-- Logs begin at Fri 2022-05-20 08:29:02 UTC, end at Fri 2022-05-20 09:33:37 UTC. -- May 20 08:58:28.296135 dv-cinder-836996-23-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 20 08:58:28.303628 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri May 20 08:58:28 2022] *** May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: nodename: dv-cinder-836996-23-iscsi May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: machine: x86_64 May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: clock source: unix May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: pcre jit disabled May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: detected number of CPU cores: 4 May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: current working directory: / May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: detected binary path: /usr/bin/uwsgi-core May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: your processes number limit is 47680 May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: your memory page size is 4096 bytes May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: detected max file descriptor number: 2048 May 20 08:58:28.305621 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: lock engine: pthread robust mutexes May 20 08:58:28.306559 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: thunder lock: enabled May 20 08:58:28.306559 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 20 08:58:28.306559 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 20 08:58:28.351365 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: Python main interpreter initialized at 0x558c668799c0 May 20 08:58:28.351365 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: python threads support enabled May 20 08:58:28.351365 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: your server socket listen backlog is limited to 100 connections May 20 08:58:28.351365 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: your mercy for graceful operations on workers is 90 seconds May 20 08:58:28.351762 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: mapped 403077 bytes (393 KB) for 2 cores May 20 08:58:28.351832 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: *** Operational MODE: preforking *** May 20 08:58:28.352104 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: *** uWSGI is running in multiple interpreter mode *** May 20 08:58:28.352104 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: spawned uWSGI master process (pid: 80624) May 20 08:58:28.352323 dv-cinder-836996-23-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 20 08:58:28.353045 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: spawned uWSGI worker 1 (pid: 80637, cores: 1) May 20 08:58:28.353203 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: spawned uWSGI worker 2 (pid: 80638, cores: 1) May 20 08:58:28.353399 dv-cinder-836996-23-iscsi devstack@keystone.service[80624]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 20 08:58:33.961838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 20 08:58:33.961838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: from cryptography.utils import int_from_bytes May 20 08:58:33.962648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 20 08:58:33.962648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: from cryptography.utils import int_from_bytes May 20 08:58:33.966406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 20 08:58:33.966406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: from cryptography.utils import int_from_bytes May 20 08:58:33.966901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 20 08:58:33.966901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: from cryptography.utils import int_from_bytes May 20 08:58:34.011974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 20 08:58:34.012150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 20 08:58:34.012346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] command line args: [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 20 08:58:34.012622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 20 08:58:34.012753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ================================================================================ {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 20 08:58:34.013043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] admin_token = **** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.013162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 20 08:58:34.013248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] config_dir = [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.013315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 20 08:58:34.013469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.013616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] command line args: [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 20 08:58:34.013701 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] config_source = [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.013769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 20 08:58:34.013846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] debug = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.013925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ================================================================================ {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 20 08:58:34.014006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] admin_token = **** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] config_dir = [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] config_source = [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] debug = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.015951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] list_limit = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016279 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] log_config_append = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] log_dir = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] log_file = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] list_limit = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] log_config_append = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.016969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] log_dir = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] log_file = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.017985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018655 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.018920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.019054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.019134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.019211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.019360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.019637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.019759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.019854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.019985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] publish_errors = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.020115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.020203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.020385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.020535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.020614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.020783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] publish_errors = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.020904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.020993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] standard_threads = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.021882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] use_journal = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] standard_threads = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022245 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] use_json = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] use_stderr = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] use_syslog = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.022890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] use_journal = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.023003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] use_json = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.023112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.023222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] use_stderr = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.023358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.023469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] use_syslog = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.023597 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.023716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 20 08:58:34.023825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.023941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.024050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.024168 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.024280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.024433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.024585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.024697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.024806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.024999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.025942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.026156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.026266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.026383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.026500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.026616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.026726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.026835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.026945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.027063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.027174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.027301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.027425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.027537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.027645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.027775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.027887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.028910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.029927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.030041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.030151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.030260 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.030368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.030561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.030674 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.030783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.030899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.031015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.031125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.031243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.031370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.031535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.031695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.031695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.031919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.032944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.033943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.034903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.035023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.035137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.035304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.035460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.035582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.035693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.035794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.035910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036471 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.036917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.037825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.038944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.039054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.039164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.039288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.039414 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.039611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.039735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.039847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.039958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.040982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.041094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.041205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.041315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.041424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.041880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.042988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.043099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.043209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.043380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.043495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.043605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.043716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.043828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.043938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.044057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.044203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.044324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.044436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.044546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.044656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.044776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.044933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045397 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.045984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046886 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.046997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.047116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.047228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.047356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.047467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.047467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.047467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.047781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.047893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] auth.external = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] auth.external = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] auth.password = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.048927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] auth.token = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] auth.password = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] auth.token = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.049932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.050955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] credential.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.051065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] credential.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.051177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.051342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.051458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.051570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.051681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.051792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.051902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.052919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.053976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.054095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.054208 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.054321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.054431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.054543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.054656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] federation.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.054775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] federation.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.054890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.055001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.055112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.055223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.055350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.055549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.055696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.055811 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.055922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056854 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.056981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.057094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.057206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.057317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.057430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.057549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.057671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.057784 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.057896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.058907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059769 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.059992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.060106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.060217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.060336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.060450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.060564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.060681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.060794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.060906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061250 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.061931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.062989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.063112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.063225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.063354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.063466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.063577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.063688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.063813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.063927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.064939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.065972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.066986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.067097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.067218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.067345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.067457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.067457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.067666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.067778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.067903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.068918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069260 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.069939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.070057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.070169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.070280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.070391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.070501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.070623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.070736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.070869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.071054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.071172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.071320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.071437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.071548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.071659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.071771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.071881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.072907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.073925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074721 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.074944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.075055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.075166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.075330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.075458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.075573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.075684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.075795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.075913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076701 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.076932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.077043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.077154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.077265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.077377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.077562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.077678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.077789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] resource.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.077900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] resource.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078242 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.078920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.079992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.080107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] role.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.080218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.080338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] role.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.080451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] role.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.080564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.080675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] role.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.080786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.080896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081575 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.081921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.082942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.083984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.084192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.084312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.084425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.084536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.084647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.084759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.084870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.084990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.085102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.085213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.085324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.085435 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.085546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.085666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.085780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.085891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086345 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.086932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.087054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.087167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.087344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.087459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.087582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.087696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.087817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.087930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] token.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] token.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.088950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.089974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.090985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.091104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.091216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.091216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.091454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.091568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.091679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.091679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.091896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.connection = **** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.connection = **** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.092923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.093948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.094969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.095081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.095193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.095316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.095428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.095619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.095736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.095847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.095959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.096070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.096182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.096306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.096420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80638) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 20 08:58:34.096532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.096532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.096532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 20 08:58:34.096532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80637) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 20 08:58:34.124771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.125030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.125254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.126580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.126815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.127046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.135546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.135841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.163594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.163793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.164240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.164475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.216915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.217723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.267113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.267325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.267681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.267917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.332005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.332150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.332315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.332426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.332548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.332638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.332783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.332905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.333005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.333073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.333278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.333409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.356153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.356417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.356562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.356649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.356725 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.356803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.356878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.356964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.357163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.357288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.357424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.357543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.357642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.357720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.368609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.368903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.381132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.381329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.381583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.381788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.381999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.382204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.382411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80638) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.383492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.383752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.383999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.384220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.384423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.384638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.384853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80637) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 20 08:58:34.399576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.399783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 20 08:58:34.400018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.400160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.400240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 20 08:58:34.400304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 20 08:58:34.400536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.400659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.400736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 20 08:58:34.400813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.401010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.401136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.401214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.401276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.401515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.401767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.412810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.413086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.413355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.413636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.414141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.414415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.414676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.414953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.422187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.423469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.425496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.425775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.426059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.426453 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.426711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.426839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.427026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.427155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.427323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.427406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.427489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.427688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.427813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.427948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.428030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.428120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.428310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.428439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.428673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.428939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.429209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.429454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.454821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.455800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.456471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.456749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.457398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.457674 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.462084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.462367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.462646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.462966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.463239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.463561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.469988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.470265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.470896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.471171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.474896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.475182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.475472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.475771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.475918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.476056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.476137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.476310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.476436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.476585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.476667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.476825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.476952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.477090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.477337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.477645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.495517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.495796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.496959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.497229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.500336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.501757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.504154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.504449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.505481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.505770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.509499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.511054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.513118 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.514637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.516989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.517288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.517571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.517857 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.518142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.518358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.518445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.518621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.518962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.519289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.519684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.519971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.537413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.538024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.538024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.538308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.539292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.539680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.539944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.540262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.546653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.548417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.548540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.548738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.550192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.550460 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.551780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.552061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.552362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.552644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.552921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.553508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.553785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.554058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.554360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.554631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.566486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.566772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.567047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.567348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.567635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.567909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.568035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.568175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.568257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.568568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.568844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.569103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.569379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.569642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.586313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.587745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.589578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.590995 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.592958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.594304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.594590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.595923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.596268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.596557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.596838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.597566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.597846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.598129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.604330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.605590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.605964 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.607195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.609219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.609517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.609798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.610077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.610715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.610995 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.611286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.611572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.619539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.619853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.620146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.620846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.621124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.621397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.628073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80637) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.628353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.628632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.628918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.629197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.629325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80638) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 20 08:58:34.629480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.629620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.629736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.629829 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.630034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.630159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.630323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.630497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.630621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.630753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.630936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.631067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.631151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.631302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.631385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.631474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.631551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.631633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80637) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.631851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.632117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.632373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.632633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80638) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 20 08:58:34.670630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558c668799c0 pid: 80637 (default app) May 20 08:58:34.671362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x558c668799c0 pid: 80638 (default app) May 20 08:58:34.674016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be02d2-d434-4ac9-9634-5fd15152a67b None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:34.674312 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be02d2-d434-4ac9-9634-5fd15152a67b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:34.674598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2be02d2-d434-4ac9-9634-5fd15152a67b None None] PATH_INFO: `/v3` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:34.678848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1/1] 10.222.0.69 () {50 vars in 791 bytes} [Fri May 20 08:58:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:42.650557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63693cb-c2cb-4a77-8711-ed65a7e4061d None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:42.651140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63693cb-c2cb-4a77-8711-ed65a7e4061d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:42.651140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63693cb-c2cb-4a77-8711-ed65a7e4061d None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:42.658320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1/2] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:42 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:42.666199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:42.667030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:42.667121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:42.734187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: INFO dbcounter [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] Registered counter for database keystone May 20 08:58:42.759056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG oslo_db.sqlalchemy.engines [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80637) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 20 08:58:42.770933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writer thread running {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 20 08:58:42.953736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG passlib.handlers.bcrypt [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80637) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 20 08:58:42.954045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG passlib.handlers.bcrypt [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80637) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 20 08:58:43.012391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:43.014005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:43.143241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 2/3] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 478 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) May 20 08:58:43.322084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63693cb-c2cb-4a77-8711-ed65a7e4061d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:43.395892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: INFO dbcounter [None req-f63693cb-c2cb-4a77-8711-ed65a7e4061d None None] Registered counter for database keystone May 20 08:58:43.417642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG oslo_db.sqlalchemy.engines [None req-f63693cb-c2cb-4a77-8711-ed65a7e4061d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80638) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 20 08:58:43.428960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writer thread running {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 20 08:58:43.547459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63693cb-c2cb-4a77-8711-ed65a7e4061d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:43.552890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38fadde-ffb9-43e3-ba13-3c9701f70955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:43.553565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fadde-ffb9-43e3-ba13-3c9701f70955 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:43.553848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fadde-ffb9-43e3-ba13-3c9701f70955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:43.554122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38fadde-ffb9-43e3-ba13-3c9701f70955 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:43.563251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fadde-ffb9-43e3-ba13-3c9701f70955 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:43.614040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d38fadde-ffb9-43e3-ba13-3c9701f70955 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 20 08:58:43.614300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38fadde-ffb9-43e3-ba13-3c9701f70955 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:43.624541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 2/4] 10.222.0.69 () {58 vars in 1324 bytes} [Fri May 20 08:58:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 08:58:43.631685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:43.648115 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceae11b3-bfe6-4578-ae1b-2fd68137e382 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:43.654618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cf611-c9d4-40a2-a26d-300be6362eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:43.655776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cf611-c9d4-40a2-a26d-300be6362eda admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:43.656077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cf611-c9d4-40a2-a26d-300be6362eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:43.656353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cf611-c9d4-40a2-a26d-300be6362eda admin admin] PATH_INFO: `/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:43.658090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cf611-c9d4-40a2-a26d-300be6362eda admin admin] RBAC: Authorizing `identity:get_project(project_id=aac440dc4ce14fa1a6d365f33d6d214c)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:43.820882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c51cf611-c9d4-40a2-a26d-300be6362eda admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 20 08:58:43.821583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51cf611-c9d4-40a2-a26d-300be6362eda admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:43.826028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 3/5] 10.222.0.69 () {56 vars in 1207 bytes} [Fri May 20 08:58:43 2022] GET /identity/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c => generated 344 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:46.023847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9d3f8-159a-4749-9265-0adb256b8f48 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:46.024479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9d3f8-159a-4749-9265-0adb256b8f48 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:46.024479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9d3f8-159a-4749-9265-0adb256b8f48 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:46.028525 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 3/6] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:58:46.034591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e856fe-fca4-423e-9b72-252889e51d72 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:46.035015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e856fe-fca4-423e-9b72-252889e51d72 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:46.035100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e856fe-fca4-423e-9b72-252889e51d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:46.056476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-97e856fe-fca4-423e-9b72-252889e51d72 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:46.058008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-97e856fe-fca4-423e-9b72-252889e51d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:46.068318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 4/7] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 20 08:58:46.091088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9d3f8-159a-4749-9265-0adb256b8f48 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:46.105375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9d3f8-159a-4749-9265-0adb256b8f48 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:46.110704 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6628279-2834-41c5-8a4f-9caab447984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:46.111487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6628279-2834-41c5-8a4f-9caab447984a admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:46.111777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6628279-2834-41c5-8a4f-9caab447984a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:46.112040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6628279-2834-41c5-8a4f-9caab447984a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:46.120408 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6628279-2834-41c5-8a4f-9caab447984a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:46.122192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6628279-2834-41c5-8a4f-9caab447984a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:46.131367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 4/8] 10.222.0.69 () {58 vars in 1324 bytes} [Fri May 20 08:58:46 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 08:58:46.137375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e856fe-fca4-423e-9b72-252889e51d72 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:46.151382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e856fe-fca4-423e-9b72-252889e51d72 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:46.157056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98588537-cfd2-49ea-a3c5-04bc33506394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:46.157922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98588537-cfd2-49ea-a3c5-04bc33506394 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:46.158206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98588537-cfd2-49ea-a3c5-04bc33506394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:46.158478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98588537-cfd2-49ea-a3c5-04bc33506394 admin admin] PATH_INFO: `/v3/users/ce1d4da72fcf4acabf4a7749465fe917` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:46.160055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98588537-cfd2-49ea-a3c5-04bc33506394 admin admin] RBAC: Authorizing `identity:get_user(user_id=ce1d4da72fcf4acabf4a7749465fe917)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:46.161899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98588537-cfd2-49ea-a3c5-04bc33506394 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:46.163491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 5/9] 10.222.0.69 () {56 vars in 1198 bytes} [Fri May 20 08:58:46 2022] GET /identity/v3/users/ce1d4da72fcf4acabf4a7749465fe917 => generated 247 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:48.424282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247aa75-78f2-4ddd-9e4c-3a88dd607d54 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:48.424824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247aa75-78f2-4ddd-9e4c-3a88dd607d54 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:48.424824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247aa75-78f2-4ddd-9e4c-3a88dd607d54 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:48.426199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 5/10] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:48.431618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d15166-afbe-4f75-b298-87f9a6e79f6e None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:48.432046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d15166-afbe-4f75-b298-87f9a6e79f6e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:48.432286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d15166-afbe-4f75-b298-87f9a6e79f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:48.449616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-d7d15166-afbe-4f75-b298-87f9a6e79f6e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:48.450797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d7d15166-afbe-4f75-b298-87f9a6e79f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:48.452907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0355d-4f11-4e3b-acb7-3cfc793d57ba None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:48.453197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0355d-4f11-4e3b-acb7-3cfc793d57ba None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:48.453452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f0355d-4f11-4e3b-acb7-3cfc793d57ba None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:48.454820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 6/11] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:58:48.459748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:48.460041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:48.460323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:48.500194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG passlib.handlers.bcrypt [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80638) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 20 08:58:48.500623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG passlib.handlers.bcrypt [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80638) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 20 08:58:48.535427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 6/12] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:58:48.546775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:48.548683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:48.558324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d15166-afbe-4f75-b298-87f9a6e79f6e None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:48.583372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:48.646311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 7/13] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:58:48.667622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d15166-afbe-4f75-b298-87f9a6e79f6e None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:48.670563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:48.703066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb92f0-87f1-4fcc-8a0a-35c9f704c603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:48.703938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb92f0-87f1-4fcc-8a0a-35c9f704c603 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:48.704293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb92f0-87f1-4fcc-8a0a-35c9f704c603 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:48.704631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fb92f0-87f1-4fcc-8a0a-35c9f704c603 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:48.717204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fb92f0-87f1-4fcc-8a0a-35c9f704c603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-c0fb92f0-87f1-4fcc-8a0a-35c9f704c603 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:58:48.728435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:58:48.729410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:58:48.730254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 7/14] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:58:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:58:48.735481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb92f0-87f1-4fcc-8a0a-35c9f704c603 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:48.774223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640cc9c-9c38-4f62-850b-d36683b1b82a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:48.776607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:48.812858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee1e7c-0e53-45c0-aa0f-e07ff800d4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:48.813852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee1e7c-0e53-45c0-aa0f-e07ff800d4de None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:48.814145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee1e7c-0e53-45c0-aa0f-e07ff800d4de None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:48.814429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daee1e7c-0e53-45c0-aa0f-e07ff800d4de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:48.817877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:48.837373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fb92f0-87f1-4fcc-8a0a-35c9f704c603 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:48.870762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:48.871460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:48.871745 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:48.872033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:48.872796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:58:48.873106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:48.874519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:48.880906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee1e7c-0e53-45c0-aa0f-e07ff800d4de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:48.882429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daee1e7c-0e53-45c0-aa0f-e07ff800d4de None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:48.886413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 8/15] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:58:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:58:48.891952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:48.981291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 8/16] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 20 08:58:48.987082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee1e7c-0e53-45c0-aa0f-e07ff800d4de None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:48.989779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:48.990800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a197aeb-f04a-4cdb-b5f3-3b46bcacd67c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:49.024005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c78d1-f26a-4fb6-9fc9-1c22973720cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:49.024952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c78d1-f26a-4fb6-9fc9-1c22973720cc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:49.025240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c78d1-f26a-4fb6-9fc9-1c22973720cc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:49.025521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c78d1-f26a-4fb6-9fc9-1c22973720cc None admin] PATH_INFO: `/v3/users/ce1d4da72fcf4acabf4a7749465fe917` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:49.026754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c78d1-f26a-4fb6-9fc9-1c22973720cc None admin] RBAC: Authorizing `identity:get_user(user_id=ce1d4da72fcf4acabf4a7749465fe917)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:49.028452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c78d1-f26a-4fb6-9fc9-1c22973720cc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:49.030081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 9/17] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:58:48 2022] GET /identity/v3/users/ce1d4da72fcf4acabf4a7749465fe917 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:58:49.035425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c78d1-f26a-4fb6-9fc9-1c22973720cc None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:49.050924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:49.083505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daee1e7c-0e53-45c0-aa0f-e07ff800d4de None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:49.117784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:49.118646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:49.118929 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:49.119226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:49.137055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c78d1-f26a-4fb6-9fc9-1c22973720cc None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:49.138081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:49.139603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 08:58:49.160389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:58:49.161404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:58:49.162423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 08:58:49.162423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 08:58:49.162423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:58:49.162423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:58:49.162423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 9/18] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:58:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:58:49.169038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:49.170140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f64b1f-7dbb-4001-9bbd-69328f74ea4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:49.170943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f64b1f-7dbb-4001-9bbd-69328f74ea4a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:49.171225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f64b1f-7dbb-4001-9bbd-69328f74ea4a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:49.171531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f64b1f-7dbb-4001-9bbd-69328f74ea4a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:49.172712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f64b1f-7dbb-4001-9bbd-69328f74ea4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:49.174169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f64b1f-7dbb-4001-9bbd-69328f74ea4a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:49.175792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 10/19] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:58:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:58:49.181341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f64b1f-7dbb-4001-9bbd-69328f74ea4a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:49.227304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:49.258429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:49.270892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe3cf37-6e04-4a86-bc63-ba1cbfbed000 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:49.279774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f64b1f-7dbb-4001-9bbd-69328f74ea4a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:49.316591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d7ab69-1462-4bbf-b1c3-00ce37226d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:49.317290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7ab69-1462-4bbf-b1c3-00ce37226d7b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:49.317576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7ab69-1462-4bbf-b1c3-00ce37226d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:49.317854 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d7ab69-1462-4bbf-b1c3-00ce37226d7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:49.318716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7ab69-1462-4bbf-b1c3-00ce37226d7b None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.domain.id=default, user.id=ce1d4da72fcf4acabf4a7749465fe917) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:58:49.319031 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7ab69-1462-4bbf-b1c3-00ce37226d7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:49.320555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d7ab69-1462-4bbf-b1c3-00ce37226d7b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:49.328352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 11/20] 10.222.0.69 () {56 vars in 1322 bytes} [Fri May 20 08:58:49 2022] GET /identity/v3/role_assignments?user.id=ce1d4da72fcf4acabf4a7749465fe917&scope.domain.id=default&role.id=6dca675ade814980a252f800fb4fa3ad => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:58:49.422831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0a429a-3aa6-42ef-ab9a-099096154a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:49.423577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a429a-3aa6-42ef-ab9a-099096154a4d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:49.423861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a429a-3aa6-42ef-ab9a-099096154a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:49.424141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0a429a-3aa6-42ef-ab9a-099096154a4d None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:49.424894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a429a-3aa6-42ef-ab9a-099096154a4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:58:49.425467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a429a-3aa6-42ef-ab9a-099096154a4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:49.426655 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0a429a-3aa6-42ef-ab9a-099096154a4d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:49.446803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 10/21] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 08:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:58:51.880251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051078a3-829e-41c0-8b8b-ba14f6d74ab3 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:51.880765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051078a3-829e-41c0-8b8b-ba14f6d74ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:51.880864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051078a3-829e-41c0-8b8b-ba14f6d74ab3 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:51.882142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 12/22] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:51.887904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dff90-1eb2-4ba9-8be6-e00a5a94e982 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:51.888729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dff90-1eb2-4ba9-8be6-e00a5a94e982 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:51.889245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754dff90-1eb2-4ba9-8be6-e00a5a94e982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:51.906845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-754dff90-1eb2-4ba9-8be6-e00a5a94e982 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:51.908430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-754dff90-1eb2-4ba9-8be6-e00a5a94e982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:51.989439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 11/23] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:58:52.027516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051078a3-829e-41c0-8b8b-ba14f6d74ab3 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:52.163944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051078a3-829e-41c0-8b8b-ba14f6d74ab3 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:52.182544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7349ab6-fc7b-4508-a4ad-0364f2c7073b None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.183160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7349ab6-fc7b-4508-a4ad-0364f2c7073b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.183754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7349ab6-fc7b-4508-a4ad-0364f2c7073b None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.185337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 12/24] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:58:52.190497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96405bab-9fdd-42d6-8382-1ec2150d4096 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.191033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96405bab-9fdd-42d6-8382-1ec2150d4096 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.191540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96405bab-9fdd-42d6-8382-1ec2150d4096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.208412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e80964-e3e2-4120-85b1-8ad7f6730951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:52.209225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80964-e3e2-4120-85b1-8ad7f6730951 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.209510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80964-e3e2-4120-85b1-8ad7f6730951 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.209785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e80964-e3e2-4120-85b1-8ad7f6730951 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.210844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-96405bab-9fdd-42d6-8382-1ec2150d4096 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:52.212568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-96405bab-9fdd-42d6-8382-1ec2150d4096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:52.220981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e80964-e3e2-4120-85b1-8ad7f6730951 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-12e80964-e3e2-4120-85b1-8ad7f6730951 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:58:52.228904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:58:52.229911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:58:52.230845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:58:52.233254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 13/25] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:58:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:58:52.234656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee1a0d-3150-4dce-b3e4-39e6f4d4d582 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.235089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee1a0d-3150-4dce-b3e4-39e6f4d4d582 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.235542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee1a0d-3150-4dce-b3e4-39e6f4d4d582 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.237328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 14/26] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:52.241224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee1a0d-3150-4dce-b3e4-39e6f4d4d582 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:52.246781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:52.273484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:52.322657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 13/27] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:58:52.324475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c80bac-36d4-4d73-8010-de99d9d10fe1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.325135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c80bac-36d4-4d73-8010-de99d9d10fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.326270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c80bac-36d4-4d73-8010-de99d9d10fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.355126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-32c80bac-36d4-4d73-8010-de99d9d10fe1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:52.357904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-32c80bac-36d4-4d73-8010-de99d9d10fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:52.436408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee1a0d-3150-4dce-b3e4-39e6f4d4d582 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:52.489558 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 14/28] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:58:52.492099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c80bac-36d4-4d73-8010-de99d9d10fe1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:52.510366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:52.511428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.511879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.512316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.513495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:58:52.515957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:52.518719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:52.534372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 15/29] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:58:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:52.538806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:52.614007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:52.662585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59766668-de4b-439b-9543-d7904c302a7b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:52.703669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c80bac-36d4-4d73-8010-de99d9d10fe1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:52.711382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:52.715302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:52.716630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.717316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.717757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.733076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:52.735210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:52.740508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbbb3f6-21e5-49f8-88e1-a33e120b7b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:52.741184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb3f6-21e5-49f8-88e1-a33e120b7b79 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.741467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb3f6-21e5-49f8-88e1-a33e120b7b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.741742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbbb3f6-21e5-49f8-88e1-a33e120b7b79 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.742503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbb3f6-21e5-49f8-88e1-a33e120b7b79 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:52.744071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbbb3f6-21e5-49f8-88e1-a33e120b7b79 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 08:58:52.756681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:58:52.757582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:58:52.758369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 08:58:52.758369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 08:58:52.758369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:58:52.758369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:58:52.760694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 16/30] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:58:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:58:52.761729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:52.864303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08302455-ffd4-43fe-9851-9882031d93d0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:52.881603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:52.894823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 15/31] 10.222.0.69 () {60 vars in 1131 bytes} [Fri May 20 08:58:52 2022] POST /identity/v3/roles => generated 219 bytes in 403 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:58:52.896677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319706-03ff-4225-991c-1937c04e90d1 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.896947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319706-03ff-4225-991c-1937c04e90d1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.897221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81319706-03ff-4225-991c-1937c04e90d1 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.902897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 16/32] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:52.902897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319706-03ff-4225-991c-1937c04e90d1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:52.906311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc406f6d-f6f9-41d1-9e06-dfd1160d98a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:52.907591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc406f6d-f6f9-41d1-9e06-dfd1160d98a0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.908067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc406f6d-f6f9-41d1-9e06-dfd1160d98a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.908548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc406f6d-f6f9-41d1-9e06-dfd1160d98a0 None admin] PATH_INFO: `/v3/users/ce1d4da72fcf4acabf4a7749465fe917` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.909981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc406f6d-f6f9-41d1-9e06-dfd1160d98a0 None admin] RBAC: Authorizing `identity:get_user(user_id=ce1d4da72fcf4acabf4a7749465fe917)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:52.911736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc406f6d-f6f9-41d1-9e06-dfd1160d98a0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:52.913752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 17/33] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:58:52 2022] GET /identity/v3/users/ce1d4da72fcf4acabf4a7749465fe917 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:52.915553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49277dda-84b7-45ad-a687-7fa5ac73befd None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:52.916081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49277dda-84b7-45ad-a687-7fa5ac73befd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:52.916897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49277dda-84b7-45ad-a687-7fa5ac73befd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:52.934788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-49277dda-84b7-45ad-a687-7fa5ac73befd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:52.936428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-49277dda-84b7-45ad-a687-7fa5ac73befd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:53.006679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 18/34] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:58:53.007998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49277dda-84b7-45ad-a687-7fa5ac73befd None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:53.044271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81319706-03ff-4225-991c-1937c04e90d1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:53.113870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:53.115593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.116085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.116571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.117548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:58:53.118166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:53.119994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:53.134522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 17/35] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 08:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:53.135955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:53.163624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49277dda-84b7-45ad-a687-7fa5ac73befd None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:53.182439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:53.191382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:53.205727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19302e-3587-4f6a-a9f8-743012d3cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:53.206872 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19302e-3587-4f6a-a9f8-743012d3cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.207409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19302e-3587-4f6a-a9f8-743012d3cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.207871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19302e-3587-4f6a-a9f8-743012d3cdd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.209274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19302e-3587-4f6a-a9f8-743012d3cdd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:53.210930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19302e-3587-4f6a-a9f8-743012d3cdd4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:53.213272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 19/36] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:58:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:53.214366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19302e-3587-4f6a-a9f8-743012d3cdd4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:53.292192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3271c90-a574-444f-93ba-f282d7ba67f8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:53.331072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19302e-3587-4f6a-a9f8-743012d3cdd4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:53.333934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6dfdb-f738-4fa9-aab5-acf105f2a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:53.334855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6dfdb-f738-4fa9-aab5-acf105f2a240 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.335382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6dfdb-f738-4fa9-aab5-acf105f2a240 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.335848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb6dfdb-f738-4fa9-aab5-acf105f2a240 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.336881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6dfdb-f738-4fa9-aab5-acf105f2a240 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:53.338496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb6dfdb-f738-4fa9-aab5-acf105f2a240 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:53.363907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 18/37] 10.222.0.69 () {60 vars in 1131 bytes} [Fri May 20 08:58:53 2022] POST /identity/v3/roles => generated 225 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:58:53.364728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6dfdb-f738-4fa9-aab5-acf105f2a240 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:53.385307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfa182b-2329-48ee-9c76-1d62e071829d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:53.387018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa182b-2329-48ee-9c76-1d62e071829d None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.387590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa182b-2329-48ee-9c76-1d62e071829d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.388275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfa182b-2329-48ee-9c76-1d62e071829d None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.389171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa182b-2329-48ee-9c76-1d62e071829d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:53.391927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfa182b-2329-48ee-9c76-1d62e071829d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:53.481450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:53.501460 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb6dfdb-f738-4fa9-aab5-acf105f2a240 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:53.542608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d1435-e402-4cd4-8dd7-e0c4d813e9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:53.543556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1435-e402-4cd4-8dd7-e0c4d813e9d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.543840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1435-e402-4cd4-8dd7-e0c4d813e9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.544128 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d1435-e402-4cd4-8dd7-e0c4d813e9d0 None admin] PATH_INFO: `/v3/domains/default/users/ce1d4da72fcf4acabf4a7749465fe917/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.546893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d1435-e402-4cd4-8dd7-e0c4d813e9d0 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ce1d4da72fcf4acabf4a7749465fe917, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:53.548428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d1435-e402-4cd4-8dd7-e0c4d813e9d0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:53.556084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 20/38] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 08:58:53 2022] POST /identity/v3/projects => generated 301 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:58:53.573124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 19/39] 10.222.0.69 () {58 vars in 1361 bytes} [Fri May 20 08:58:53 2022] PUT /identity/v3/domains/default/users/ce1d4da72fcf4acabf4a7749465fe917/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:58:53.788463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfede8-d88b-47a2-b9c5-d4dcd1a2053c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.789151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfede8-d88b-47a2-b9c5-d4dcd1a2053c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.789651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dfede8-d88b-47a2-b9c5-d4dcd1a2053c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.791885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 21/40] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:53.797281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899444c6-1478-4d76-92f3-4a839eac6e93 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.797570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899444c6-1478-4d76-92f3-4a839eac6e93 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.797858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899444c6-1478-4d76-92f3-4a839eac6e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.818296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-899444c6-1478-4d76-92f3-4a839eac6e93 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:53.820053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-899444c6-1478-4d76-92f3-4a839eac6e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:53.856653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd87532-939e-4bc4-ab00-4c999410d2ff None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.856975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd87532-939e-4bc4-ab00-4c999410d2ff None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.857212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd87532-939e-4bc4-ab00-4c999410d2ff None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.858713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 22/41] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:53.865329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959bfe9-4a8e-486f-b35e-705a78ff5880 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:53.865622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959bfe9-4a8e-486f-b35e-705a78ff5880 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:53.865903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959bfe9-4a8e-486f-b35e-705a78ff5880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:53.888032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b959bfe9-4a8e-486f-b35e-705a78ff5880 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:53.890065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b959bfe9-4a8e-486f-b35e-705a78ff5880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:53.896026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 20/42] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:58:53.920705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899444c6-1478-4d76-92f3-4a839eac6e93 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:53.927359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:53.968582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 23/43] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:58:53.990702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959bfe9-4a8e-486f-b35e-705a78ff5880 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:54.006758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:54.032651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899444c6-1478-4d76-92f3-4a839eac6e93 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:54.070244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611f90-f31a-4637-abc3-488622da0cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:54.070922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611f90-f31a-4637-abc3-488622da0cba None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:54.071281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611f90-f31a-4637-abc3-488622da0cba None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:54.071566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2611f90-f31a-4637-abc3-488622da0cba None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:54.072325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2611f90-f31a-4637-abc3-488622da0cba None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:54.073773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2611f90-f31a-4637-abc3-488622da0cba None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:54.097696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 21/44] 10.222.0.69 () {60 vars in 1131 bytes} [Fri May 20 08:58:53 2022] POST /identity/v3/roles => generated 223 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 08:58:54.119211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959bfe9-4a8e-486f-b35e-705a78ff5880 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:54.159239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a187e55e-170c-4a75-b846-e6d5c29fb1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:54.160108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e55e-170c-4a75-b846-e6d5c29fb1ec None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:54.160394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e55e-170c-4a75-b846-e6d5c29fb1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:54.160669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a187e55e-170c-4a75-b846-e6d5c29fb1ec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:54.162074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187e55e-170c-4a75-b846-e6d5c29fb1ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:54.163805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a187e55e-170c-4a75-b846-e6d5c29fb1ec None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:54.170703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 24/45] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:58:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:58:54.172786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611f90-f31a-4637-abc3-488622da0cba None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:54.286815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2611f90-f31a-4637-abc3-488622da0cba None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:54.322629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9e7a5-3f2b-4ce0-ac5e-af657241f78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:54.323711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e7a5-3f2b-4ce0-ac5e-af657241f78e None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:54.323814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e7a5-3f2b-4ce0-ac5e-af657241f78e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:54.324085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e7a5-3f2b-4ce0-ac5e-af657241f78e None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:54.324848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9e7a5-3f2b-4ce0-ac5e-af657241f78e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:54.326331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9e7a5-3f2b-4ce0-ac5e-af657241f78e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:54.333542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:54.360062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 22/46] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 08:58:54 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:58:55.852038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05004a7-487b-489a-94f9-02efc3aabd36 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:55.852553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05004a7-487b-489a-94f9-02efc3aabd36 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:55.852671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05004a7-487b-489a-94f9-02efc3aabd36 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:55.854168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 25/47] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:55.859317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f66c9-1010-46fc-91b4-95865658f1a6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:55.859728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f66c9-1010-46fc-91b4-95865658f1a6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:55.859883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f66c9-1010-46fc-91b4-95865658f1a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:55.877503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-0f2f66c9-1010-46fc-91b4-95865658f1a6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:55.878687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0f2f66c9-1010-46fc-91b4-95865658f1a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:55.947960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 23/48] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:58:55.971055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05004a7-487b-489a-94f9-02efc3aabd36 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:56.007653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:56.077123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05004a7-487b-489a-94f9-02efc3aabd36 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:56.112832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cb8a9-e37b-4c8e-80e7-741e703181fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:56.113643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cb8a9-e37b-4c8e-80e7-741e703181fa None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.113936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cb8a9-e37b-4c8e-80e7-741e703181fa None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.114216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1cb8a9-e37b-4c8e-80e7-741e703181fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.125218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1cb8a9-e37b-4c8e-80e7-741e703181fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-8e1cb8a9-e37b-4c8e-80e7-741e703181fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:58:56.132861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:58:56.133970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:58:56.134752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 26/49] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:58:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:58:56.140144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f66c9-1010-46fc-91b4-95865658f1a6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:56.243917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f66c9-1010-46fc-91b4-95865658f1a6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:56.262319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:56.280285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:56.280960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.281241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.281515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.282282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:58:56.282591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:56.284028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:56.296336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 24/50] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:58:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:56.301906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cb8a9-e37b-4c8e-80e7-741e703181fa None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:56.404416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:56.412315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1cb8a9-e37b-4c8e-80e7-741e703181fa None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:56.448103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709e6ec-4516-41ae-969f-28940def9934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:56.448987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e6ec-4516-41ae-969f-28940def9934 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.449272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e6ec-4516-41ae-969f-28940def9934 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.449549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709e6ec-4516-41ae-969f-28940def9934 None admin] PATH_INFO: `/v3/users/ce1d4da72fcf4acabf4a7749465fe917` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.450771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709e6ec-4516-41ae-969f-28940def9934 None admin] RBAC: Authorizing `identity:get_user(user_id=ce1d4da72fcf4acabf4a7749465fe917)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:56.452277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709e6ec-4516-41ae-969f-28940def9934 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:56.453948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 27/51] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:58:56 2022] GET /identity/v3/users/ce1d4da72fcf4acabf4a7749465fe917 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:56.459519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:56.564547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e87f5-5a46-41fe-961e-f8c106749a4b None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.564790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e87f5-5a46-41fe-961e-f8c106749a4b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.565082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e87f5-5a46-41fe-961e-f8c106749a4b None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.566445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 28/52] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:56.567173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d410d0e-a922-4d9c-beb0-103da1e72970 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:56.571564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b193b-ae88-4a21-9a6a-cdce3f42a00d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.571859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b193b-ae88-4a21-9a6a-cdce3f42a00d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.572176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b193b-ae88-4a21-9a6a-cdce3f42a00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.590503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-d84b193b-ae88-4a21-9a6a-cdce3f42a00d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:56.591731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d84b193b-ae88-4a21-9a6a-cdce3f42a00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:56.600297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50bbf8-c3d3-496b-8663-cdbc07c95f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:56.601097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50bbf8-c3d3-496b-8663-cdbc07c95f7a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.601383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50bbf8-c3d3-496b-8663-cdbc07c95f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.601662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50bbf8-c3d3-496b-8663-cdbc07c95f7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.602862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50bbf8-c3d3-496b-8663-cdbc07c95f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:56.604344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50bbf8-c3d3-496b-8663-cdbc07c95f7a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:56.606047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 25/53] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:58:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:56.611739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50bbf8-c3d3-496b-8663-cdbc07c95f7a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:56.657501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 29/54] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:58:56.658680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:56.681565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b193b-ae88-4a21-9a6a-cdce3f42a00d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:56.709481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50bbf8-c3d3-496b-8663-cdbc07c95f7a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:56.744987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:56.745651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.745945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.746250 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.747095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.domain.id=default, user.id=ce1d4da72fcf4acabf4a7749465fe917) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:58:56.747428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:56.748889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:56.757161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 26/55] 10.222.0.69 () {56 vars in 1322 bytes} [Fri May 20 08:58:56 2022] GET /identity/v3/role_assignments?user.id=ce1d4da72fcf4acabf4a7749465fe917&scope.domain.id=default&role.id=6dca675ade814980a252f800fb4fa3ad => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:58:56.778000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:56.787791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b193b-ae88-4a21-9a6a-cdce3f42a00d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:56.826015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bf985-f46d-402e-b7c4-b7d854641ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:56.826848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf985-f46d-402e-b7c4-b7d854641ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.827127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf985-f46d-402e-b7c4-b7d854641ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.827414 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bf985-f46d-402e-b7c4-b7d854641ae4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.828706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bf985-f46d-402e-b7c4-b7d854641ae4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:56.830207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bf985-f46d-402e-b7c4-b7d854641ae4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:56.832012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 30/56] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:58:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:56.838094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:56.945662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4842835d-4f49-40b2-95ed-34b60ac7d37a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:56.960073 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:56.983277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddec26e-79cf-45fe-bf85-d7ceee43c1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:56.984060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddec26e-79cf-45fe-bf85-d7ceee43c1e5 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:56.984342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddec26e-79cf-45fe-bf85-d7ceee43c1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:56.984607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddec26e-79cf-45fe-bf85-d7ceee43c1e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:56.985335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddec26e-79cf-45fe-bf85-d7ceee43c1e5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:56.986795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddec26e-79cf-45fe-bf85-d7ceee43c1e5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:57.012752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 27/57] 10.222.0.69 () {60 vars in 1140 bytes} [Fri May 20 08:58:56 2022] POST /identity/v3/projects => generated 298 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:58:59.196253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207a07-d2dd-493b-9c3c-9faa2f6f45f0 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:59.197032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207a07-d2dd-493b-9c3c-9faa2f6f45f0 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:59.197032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51207a07-d2dd-493b-9c3c-9faa2f6f45f0 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:59.199209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 31/58] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:58:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:58:59.207991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf828-ecb6-4994-ada1-5ed6c4d9ce61 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:59.208397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf828-ecb6-4994-ada1-5ed6c4d9ce61 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:59.208601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbf828-ecb6-4994-ada1-5ed6c4d9ce61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:59.233484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-74fbf828-ecb6-4994-ada1-5ed6c4d9ce61 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:58:59.234683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-74fbf828-ecb6-4994-ada1-5ed6c4d9ce61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:58:59.306426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 28/59] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:58:59.331094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51207a07-d2dd-493b-9c3c-9faa2f6f45f0 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:59.437420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51207a07-d2dd-493b-9c3c-9faa2f6f45f0 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:59.471906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce07fe72-5a80-4435-a4f1-c096272e179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:59.472628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07fe72-5a80-4435-a4f1-c096272e179d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:59.472919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07fe72-5a80-4435-a4f1-c096272e179d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:59.473196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce07fe72-5a80-4435-a4f1-c096272e179d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:59.474412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07fe72-5a80-4435-a4f1-c096272e179d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:59.475921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce07fe72-5a80-4435-a4f1-c096272e179d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:59.477609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 32/60] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:58:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:58:59.483882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbf828-ecb6-4994-ada1-5ed6c4d9ce61 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:58:59.585101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:58:59.593534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbf828-ecb6-4994-ada1-5ed6c4d9ce61 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:58:59.627907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730f2a5d-6e55-4891-8095-fed13ac7c30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:58:59.628562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f2a5d-6e55-4891-8095-fed13ac7c30d None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:58:59.628844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f2a5d-6e55-4891-8095-fed13ac7c30d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:58:59.629115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730f2a5d-6e55-4891-8095-fed13ac7c30d None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:58:59.629819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f2a5d-6e55-4891-8095-fed13ac7c30d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:58:59.631296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730f2a5d-6e55-4891-8095-fed13ac7c30d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:58:59.658966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 29/61] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:58:59 2022] POST /identity/v3/users => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:02.899724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02821486-865a-48ce-8bbd-25e42683589c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:02.900765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02821486-865a-48ce-8bbd-25e42683589c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:02.901283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02821486-865a-48ce-8bbd-25e42683589c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:02.902978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 33/62] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:02.909020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e8a5c-530b-4750-8777-c9aa8102d32c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:02.909492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e8a5c-530b-4750-8777-c9aa8102d32c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:02.909592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e8a5c-530b-4750-8777-c9aa8102d32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:02.931511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-eb2e8a5c-530b-4750-8777-c9aa8102d32c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:02.931511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-eb2e8a5c-530b-4750-8777-c9aa8102d32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:03.033832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 30/63] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:03.036553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a26eb-fcad-457d-99b4-e46b6effc9bf None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.036849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a26eb-fcad-457d-99b4-e46b6effc9bf None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.037135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a26eb-fcad-457d-99b4-e46b6effc9bf None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.038614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 34/64] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:03.055883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b8481-2ce6-45f1-81eb-7d342010e894 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.056176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b8481-2ce6-45f1-81eb-7d342010e894 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.056461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3b8481-2ce6-45f1-81eb-7d342010e894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.063343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a26eb-fcad-457d-99b4-e46b6effc9bf None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.081205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5c3b8481-2ce6-45f1-81eb-7d342010e894 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:03.082486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5c3b8481-2ce6-45f1-81eb-7d342010e894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:03.092623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:03.188496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 31/65] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:03.190105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e85a9-833d-4bff-8663-bd9a4ff5a06d None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.190529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e85a9-833d-4bff-8663-bd9a4ff5a06d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.190943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e85a9-833d-4bff-8663-bd9a4ff5a06d None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.192487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 32/66] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:03.198860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0208d-a4a9-4af0-980a-b9ef83865bd7 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.199456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0208d-a4a9-4af0-980a-b9ef83865bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.199878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0208d-a4a9-4af0-980a-b9ef83865bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.213034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a26eb-fcad-457d-99b4-e46b6effc9bf None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:03.218553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d7b0208d-a4a9-4af0-980a-b9ef83865bd7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:03.220933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d7b0208d-a4a9-4af0-980a-b9ef83865bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:03.224369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:03.256754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3ae99-0f07-4899-bb79-786565d0a8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:03.257811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3ae99-0f07-4899-bb79-786565d0a8fe None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.258267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3ae99-0f07-4899-bb79-786565d0a8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.258814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3ae99-0f07-4899-bb79-786565d0a8fe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.270354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3ae99-0f07-4899-bb79-786565d0a8fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-94c3ae99-0f07-4899-bb79-786565d0a8fe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:03.278641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:03.279754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:03.280543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:03.282615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 35/67] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:03.288302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3ae99-0f07-4899-bb79-786565d0a8fe None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.301828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 33/68] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:03.303826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0208d-a4a9-4af0-980a-b9ef83865bd7 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.402390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:03.415667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0208d-a4a9-4af0-980a-b9ef83865bd7 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:03.415839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3ae99-0f07-4899-bb79-786565d0a8fe None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:03.452817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78913c81-8f54-4558-b5f5-8660d7f5f792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:03.454120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c81-8f54-4558-b5f5-8660d7f5f792 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.454554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c81-8f54-4558-b5f5-8660d7f5f792 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.454968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c81-8f54-4558-b5f5-8660d7f5f792 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.460668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:03.461353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.461717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.461971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.465159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:03.465485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:03.466994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:03.467160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78913c81-8f54-4558-b5f5-8660d7f5f792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-78913c81-8f54-4558-b5f5-8660d7f5f792 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:03.475344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:03.476353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:03.477373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:03.478170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 34/69] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:03.478170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.484289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 36/70] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:03.485319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78913c81-8f54-4558-b5f5-8660d7f5f792 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.486010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:03.591654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eaf1cb-5f3c-44ab-a698-645c056803ed None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:03.605895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78913c81-8f54-4558-b5f5-8660d7f5f792 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:03.627526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710ede-4f4b-466c-88e6-e574b7340d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:03.628580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710ede-4f4b-466c-88e6-e574b7340d1b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.629000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710ede-4f4b-466c-88e6-e574b7340d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.629418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4710ede-4f4b-466c-88e6-e574b7340d1b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.641416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d435c-9513-4e02-848d-9d8cc00c491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:03.642526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d435c-9513-4e02-848d-9d8cc00c491a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.642954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d435c-9513-4e02-848d-9d8cc00c491a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.643121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4710ede-4f4b-466c-88e6-e574b7340d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:03.643404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d435c-9513-4e02-848d-9d8cc00c491a None admin] PATH_INFO: `/v3/users/ce1d4da72fcf4acabf4a7749465fe917` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.644777 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d435c-9513-4e02-848d-9d8cc00c491a None admin] RBAC: Authorizing `identity:get_user(user_id=ce1d4da72fcf4acabf4a7749465fe917)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:03.646395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d435c-9513-4e02-848d-9d8cc00c491a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:03.648222 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 37/71] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/users/ce1d4da72fcf4acabf4a7749465fe917 => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:03.649215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d435c-9513-4e02-848d-9d8cc00c491a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b4710ede-4f4b-466c-88e6-e574b7340d1b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:03.657666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:03.658768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:03.659669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:03.661720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 35/72] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:03.662636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710ede-4f4b-466c-88e6-e574b7340d1b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.673275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:03.727081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:03.756430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d435c-9513-4e02-848d-9d8cc00c491a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:03.764693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4710ede-4f4b-466c-88e6-e574b7340d1b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:03.789174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:03.789900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.790187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.790451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.791622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:03.791938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:03.793349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:03.801354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 38/73] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:03.802103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.803983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d37495-47c1-4774-8653-e3edf36520da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:03.804819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d37495-47c1-4774-8653-e3edf36520da None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.805116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d37495-47c1-4774-8653-e3edf36520da None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.806271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d37495-47c1-4774-8653-e3edf36520da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.875788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d37495-47c1-4774-8653-e3edf36520da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:03.877427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d37495-47c1-4774-8653-e3edf36520da None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:03.878010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:03.899975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135851e5-fadf-4cee-ba60-c226243a12df None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:03.933735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0392ae6-f9f8-4d6b-a1ef-bc54c1c263c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:03.934403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392ae6-f9f8-4d6b-a1ef-bc54c1c263c1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.934692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392ae6-f9f8-4d6b-a1ef-bc54c1c263c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.934967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0392ae6-f9f8-4d6b-a1ef-bc54c1c263c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.935766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0392ae6-f9f8-4d6b-a1ef-bc54c1c263c1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:03.936075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0392ae6-f9f8-4d6b-a1ef-bc54c1c263c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:03.936558 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:03.937710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0392ae6-f9f8-4d6b-a1ef-bc54c1c263c1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:03.945469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 39/74] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:03.946797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05747b41-42f8-415c-bae8-dcc88ab9bd41 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.947085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05747b41-42f8-415c-bae8-dcc88ab9bd41 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.947386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05747b41-42f8-415c-bae8-dcc88ab9bd41 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.948767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 40/75] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:03.949691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fcfac-e9cb-42d5-852b-002fec9faa32 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.949976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fcfac-e9cb-42d5-852b-002fec9faa32 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.950252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0fcfac-e9cb-42d5-852b-002fec9faa32 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.951468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 41/76] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:03.952156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fcfac-e9cb-42d5-852b-002fec9faa32 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:03.978012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 36/77] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:03.979365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c47c1f-b5a1-4696-913a-2fce2768a7ed None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:03.979658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c47c1f-b5a1-4696-913a-2fce2768a7ed None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:03.979941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c47c1f-b5a1-4696-913a-2fce2768a7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:03.996303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e7c47c1f-b5a1-4696-913a-2fce2768a7ed None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:03.997519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e7c47c1f-b5a1-4696-913a-2fce2768a7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:04.059135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0fcfac-e9cb-42d5-852b-002fec9faa32 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.067842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 37/78] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:04.068692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c47c1f-b5a1-4696-913a-2fce2768a7ed None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.093482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51aa48f-bc64-414d-b38c-3a9e7a8c5ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.094374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa48f-bc64-414d-b38c-3a9e7a8c5ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.094660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa48f-bc64-414d-b38c-3a9e7a8c5ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.094936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51aa48f-bc64-414d-b38c-3a9e7a8c5ba2 None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.105301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:04.108718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51aa48f-bc64-414d-b38c-3a9e7a8c5ba2 None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:04.110289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51aa48f-bc64-414d-b38c-3a9e7a8c5ba2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:04.112257 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 42/79] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:03 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:04.113311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44805dd-b54b-4cab-a8db-cc8f1ba048e4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.113603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44805dd-b54b-4cab-a8db-cc8f1ba048e4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.113971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44805dd-b54b-4cab-a8db-cc8f1ba048e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.129770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f44805dd-b54b-4cab-a8db-cc8f1ba048e4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:04.130874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f44805dd-b54b-4cab-a8db-cc8f1ba048e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:04.167498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:04.177608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c47c1f-b5a1-4696-913a-2fce2768a7ed None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.196050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 43/80] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:04.196900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44805dd-b54b-4cab-a8db-cc8f1ba048e4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.226894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face78af-f85b-4b95-a807-4fb937cf39f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.227819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face78af-f85b-4b95-a807-4fb937cf39f8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.228105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face78af-f85b-4b95-a807-4fb937cf39f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.228380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face78af-f85b-4b95-a807-4fb937cf39f8 None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.229611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face78af-f85b-4b95-a807-4fb937cf39f8 None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:04.231089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-face78af-f85b-4b95-a807-4fb937cf39f8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:04.232772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 38/81] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:04.233535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face78af-f85b-4b95-a807-4fb937cf39f8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.296025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44805dd-b54b-4cab-a8db-cc8f1ba048e4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.320708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:04.329494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face78af-f85b-4b95-a807-4fb937cf39f8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.330489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151011b-2c08-4964-aab4-cc44ca7c5030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.331405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151011b-2c08-4964-aab4-cc44ca7c5030 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.331689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151011b-2c08-4964-aab4-cc44ca7c5030 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.331962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b151011b-2c08-4964-aab4-cc44ca7c5030 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.333255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151011b-2c08-4964-aab4-cc44ca7c5030 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:04.334726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b151011b-2c08-4964-aab4-cc44ca7c5030 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:04.336511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 44/82] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:04.337321 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151011b-2c08-4964-aab4-cc44ca7c5030 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.365071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4ac2b-5398-484a-971b-81c7be1d34c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.365901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ac2b-5398-484a-971b-81c7be1d34c1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.366180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ac2b-5398-484a-971b-81c7be1d34c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.366466 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4ac2b-5398-484a-971b-81c7be1d34c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.367722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4ac2b-5398-484a-971b-81c7be1d34c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:04.369207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b4ac2b-5398-484a-971b-81c7be1d34c1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:04.370820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 39/83] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:04.371598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4ac2b-5398-484a-971b-81c7be1d34c1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.404389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:04.447697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b151011b-2c08-4964-aab4-cc44ca7c5030 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.477051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4ac2b-5398-484a-971b-81c7be1d34c1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.480542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1916298-fc37-4452-8946-6559806029f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.481198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916298-fc37-4452-8946-6559806029f2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.481482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916298-fc37-4452-8946-6559806029f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.481759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1916298-fc37-4452-8946-6559806029f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.509610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52520fcf-ec14-4570-abe5-49e65b47b765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.510411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52520fcf-ec14-4570-abe5-49e65b47b765 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.510723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52520fcf-ec14-4570-abe5-49e65b47b765 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.510995 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52520fcf-ec14-4570-abe5-49e65b47b765 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.521846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52520fcf-ec14-4570-abe5-49e65b47b765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-52520fcf-ec14-4570-abe5-49e65b47b765 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:04.529533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:04.530575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:04.531399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 40/84] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:04.532049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52520fcf-ec14-4570-abe5-49e65b47b765 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.546712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:04.554221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1916298-fc37-4452-8946-6559806029f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:04.555783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1916298-fc37-4452-8946-6559806029f2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:04.621033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:04.634507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52520fcf-ec14-4570-abe5-49e65b47b765 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.652263 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 45/85] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:04.653115 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1916298-fc37-4452-8946-6559806029f2 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.673797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a28f3-c14b-4fcd-8ae4-f06f59769696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.674373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a28f3-c14b-4fcd-8ae4-f06f59769696 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.674660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a28f3-c14b-4fcd-8ae4-f06f59769696 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.674927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a28f3-c14b-4fcd-8ae4-f06f59769696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.738918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a28f3-c14b-4fcd-8ae4-f06f59769696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:04.740426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a28f3-c14b-4fcd-8ae4-f06f59769696 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:04.745551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:04.752045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1916298-fc37-4452-8946-6559806029f2 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.785355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.786017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.786314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.786575 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.787443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7, user.id=ce1d4da72fcf4acabf4a7749465fe917) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:04.787754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:04.789218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:04.795653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 46/86] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/role_assignments?user.id=ce1d4da72fcf4acabf4a7749465fe917&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=6dca675ade814980a252f800fb4fa3ad => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:04.796173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.829166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:04.845194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 41/87] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:04.850670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a28f3-c14b-4fcd-8ae4-f06f59769696 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:04.908487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95b589-9a8b-4c7c-a8b9-2ffc48cb7ca9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.945535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdecf49-3576-4866-acee-64c67c818ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:04.946155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdecf49-3576-4866-acee-64c67c818ef4 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:04.946439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdecf49-3576-4866-acee-64c67c818ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:04.946716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdecf49-3576-4866-acee-64c67c818ef4 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:04.947477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdecf49-3576-4866-acee-64c67c818ef4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:04.948970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdecf49-3576-4866-acee-64c67c818ef4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:04.955907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a28f3-c14b-4fcd-8ae4-f06f59769696 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:04.980026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 47/88] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:04 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:04.987511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdecf49-3576-4866-acee-64c67c818ef4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:05.000715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:05.001574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:05.002226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:05.002570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:05.005011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] RBAC: Adding query filter params (name=member) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:05.005011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:05.006373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:05.016543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 42/89] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:05.017216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:05.051834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:05.088726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:05.122329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdecf49-3576-4866-acee-64c67c818ef4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:05.150224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4e860-93f0-4f04-ada1-4175b5194887 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:05.160045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2233-b7db-4379-b44c-d8fc433651c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:05.161003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2233-b7db-4379-b44c-d8fc433651c7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:05.161296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2233-b7db-4379-b44c-d8fc433651c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:05.161580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d2233-b7db-4379-b44c-d8fc433651c7 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:05.162975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2233-b7db-4379-b44c-d8fc433651c7 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:05.164693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d2233-b7db-4379-b44c-d8fc433651c7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:05.166948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 48/90] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:04 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:05.167921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2233-b7db-4379-b44c-d8fc433651c7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:05.191180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88bd29-2502-4bf5-8e84-7dce95a45081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:05.192090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88bd29-2502-4bf5-8e84-7dce95a45081 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:05.192480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88bd29-2502-4bf5-8e84-7dce95a45081 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:05.192763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88bd29-2502-4bf5-8e84-7dce95a45081 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:05.193959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88bd29-2502-4bf5-8e84-7dce95a45081 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:05.195472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88bd29-2502-4bf5-8e84-7dce95a45081 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:05.197559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 43/91] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:05 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:05.198200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88bd29-2502-4bf5-8e84-7dce95a45081 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:05.267185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:05.294799 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d2233-b7db-4379-b44c-d8fc433651c7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:05.313301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88bd29-2502-4bf5-8e84-7dce95a45081 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:05.333973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:05.342605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667e9b-16b0-4bae-a8c9-2ce6cdafab9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:05.343478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667e9b-16b0-4bae-a8c9-2ce6cdafab9f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:05.343769 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667e9b-16b0-4bae-a8c9-2ce6cdafab9f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:05.344050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4667e9b-16b0-4bae-a8c9-2ce6cdafab9f None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:05.346197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4667e9b-16b0-4bae-a8c9-2ce6cdafab9f None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:05.347851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4667e9b-16b0-4bae-a8c9-2ce6cdafab9f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:05.349826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 49/92] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:05 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:05.350664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667e9b-16b0-4bae-a8c9-2ce6cdafab9f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:05.361519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:05.362186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:05.362466 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:05.362849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:05.363756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] RBAC: Adding query filter params (role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7, user.id=bdd8156d03f1489f99984f234d7d8ca7) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:05.364064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:05.365649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:05.373952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 44/93] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:05 2022] GET /identity/v3/role_assignments?user.id=bdd8156d03f1489f99984f234d7d8ca7&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:05.375930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:05.485383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4667e9b-16b0-4bae-a8c9-2ce6cdafab9f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:05.533123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8017ca02-d570-4dc8-b4ea-69f9f7a13296 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:05.560644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:05.561334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:05.561623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:05.561911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:05.562784 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] RBAC: Adding query filter params (role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7, user.id=bdd8156d03f1489f99984f234d7d8ca7) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:05.563100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:05.564633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:05.574439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e0b2-e1bd-4bcf-b989-36bd5c96a695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:05.575296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e0b2-e1bd-4bcf-b989-36bd5c96a695 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:05.575689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e0b2-e1bd-4bcf-b989-36bd5c96a695 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:05.576072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d102e0b2-e1bd-4bcf-b989-36bd5c96a695 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:05.581467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 50/94] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:05 2022] GET /identity/v3/role_assignments?user.id=bdd8156d03f1489f99984f234d7d8ca7&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 259 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:05.655490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:05.684227 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102e0b2-e1bd-4bcf-b989-36bd5c96a695 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:05.685864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d102e0b2-e1bd-4bcf-b989-36bd5c96a695 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:05.809801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 45/95] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:05.815759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:05.847564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:05.952469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088c378-19a0-4ba7-9824-dc6a43677948 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:05.999989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4837d-f17f-4584-b31e-d6b4699df71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:06.000884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4837d-f17f-4584-b31e-d6b4699df71e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:06.001174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4837d-f17f-4584-b31e-d6b4699df71e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:06.001457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4837d-f17f-4584-b31e-d6b4699df71e None admin] PATH_INFO: `/v3/projects/a2ceac24453940eea8e39cedc308fd02` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:06.002711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4837d-f17f-4584-b31e-d6b4699df71e None admin] RBAC: Authorizing `identity:get_project(project_id=a2ceac24453940eea8e39cedc308fd02)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:06.004229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4837d-f17f-4584-b31e-d6b4699df71e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:06.006142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 51/96] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:05 2022] GET /identity/v3/projects/a2ceac24453940eea8e39cedc308fd02 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:06.012238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e0b2-e1bd-4bcf-b989-36bd5c96a695 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:06.135372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:06.157204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d102e0b2-e1bd-4bcf-b989-36bd5c96a695 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:06.216548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0478a70-10fa-4cf6-bab9-0ea69e68c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:06.217229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0478a70-10fa-4cf6-bab9-0ea69e68c174 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:06.217514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0478a70-10fa-4cf6-bab9-0ea69e68c174 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:06.217794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0478a70-10fa-4cf6-bab9-0ea69e68c174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:06.218646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0478a70-10fa-4cf6-bab9-0ea69e68c174 None admin] RBAC: Adding query filter params (role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=a2ceac24453940eea8e39cedc308fd02, user.id=bdd8156d03f1489f99984f234d7d8ca7) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:06.218958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0478a70-10fa-4cf6-bab9-0ea69e68c174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:06.220447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0478a70-10fa-4cf6-bab9-0ea69e68c174 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:06.226347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 46/97] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:06 2022] GET /identity/v3/role_assignments?user.id=bdd8156d03f1489f99984f234d7d8ca7&scope.project.id=a2ceac24453940eea8e39cedc308fd02&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:08.028279 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc6a8-d2a5-4a33-b4f8-7be0e219053f None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.029512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc6a8-d2a5-4a33-b4f8-7be0e219053f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.030010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cdc6a8-d2a5-4a33-b4f8-7be0e219053f None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.031929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 52/98] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:08.036865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d163-ed2b-4414-b493-c33700bfc7c8 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.037712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d163-ed2b-4414-b493-c33700bfc7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.038202 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd70d163-ed2b-4414-b493-c33700bfc7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.064376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-cd70d163-ed2b-4414-b493-c33700bfc7c8 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:08.065947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cd70d163-ed2b-4414-b493-c33700bfc7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:08.153600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 47/99] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:08.195996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc6a8-d2a5-4a33-b4f8-7be0e219053f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:08.298982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:08.319348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cdc6a8-d2a5-4a33-b4f8-7be0e219053f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:08.363055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad27d3-2669-41fe-85be-92e039866c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:08.363891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad27d3-2669-41fe-85be-92e039866c49 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.364180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad27d3-2669-41fe-85be-92e039866c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.364461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad27d3-2669-41fe-85be-92e039866c49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.374756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15dd07-d9bd-4461-8786-9cd77376f986 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.375049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15dd07-d9bd-4461-8786-9cd77376f986 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.375359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15dd07-d9bd-4461-8786-9cd77376f986 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.377000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 48/100] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:08.379951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad27d3-2669-41fe-85be-92e039866c49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:08.385095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bc1ae-8502-4948-8abc-4cb922fefcf9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.385388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bc1ae-8502-4948-8abc-4cb922fefcf9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.385672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bc1ae-8502-4948-8abc-4cb922fefcf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-79ad27d3-2669-41fe-85be-92e039866c49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:08.387658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:08.388536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:08.389718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 53/101] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:08.394558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad27d3-2669-41fe-85be-92e039866c49 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:08.416615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f14bc1ae-8502-4948-8abc-4cb922fefcf9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:08.417846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f14bc1ae-8502-4948-8abc-4cb922fefcf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:08.563671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 49/102] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:08.573481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad27d3-2669-41fe-85be-92e039866c49 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:08.613616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bc1ae-8502-4948-8abc-4cb922fefcf9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:08.624551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:08.633897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61675d69-6183-4439-a679-0d9395956726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:08.634915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61675d69-6183-4439-a679-0d9395956726 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.635356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61675d69-6183-4439-a679-0d9395956726 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.635537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61675d69-6183-4439-a679-0d9395956726 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.636286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61675d69-6183-4439-a679-0d9395956726 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:08.637607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61675d69-6183-4439-a679-0d9395956726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:08.639077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61675d69-6183-4439-a679-0d9395956726 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:08.663661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 54/103] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:08.671558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61675d69-6183-4439-a679-0d9395956726 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:08.683788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:08.794817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bc1ae-8502-4948-8abc-4cb922fefcf9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:08.826263 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61675d69-6183-4439-a679-0d9395956726 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:08.832109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f4c18-0784-4f47-8d23-13f6f7394df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:08.833154 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4c18-0784-4f47-8d23-13f6f7394df9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.833629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4c18-0784-4f47-8d23-13f6f7394df9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.834155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f4c18-0784-4f47-8d23-13f6f7394df9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.845986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f4c18-0784-4f47-8d23-13f6f7394df9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-e02f4c18-0784-4f47-8d23-13f6f7394df9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:08.857469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:08.858507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:08.859527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:08.860650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097cde5-3a2b-481e-a27a-34ebda255eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:08.861533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cde5-3a2b-481e-a27a-34ebda255eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.861817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cde5-3a2b-481e-a27a-34ebda255eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.861992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 50/104] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:08.862089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097cde5-3a2b-481e-a27a-34ebda255eb2 None admin] PATH_INFO: `/v3/users/ce1d4da72fcf4acabf4a7749465fe917` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.863281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097cde5-3a2b-481e-a27a-34ebda255eb2 None admin] RBAC: Authorizing `identity:get_user(user_id=ce1d4da72fcf4acabf4a7749465fe917)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:08.863738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74897-7766-4d0c-bcbd-0544ccc7cddf None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.864164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74897-7766-4d0c-bcbd-0544ccc7cddf None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.864681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af74897-7766-4d0c-bcbd-0544ccc7cddf None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.864786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097cde5-3a2b-481e-a27a-34ebda255eb2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:08.866191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 51/105] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:08.874147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 55/106] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:08 2022] GET /identity/v3/users/ce1d4da72fcf4acabf4a7749465fe917 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:08.874147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb358d-2210-46bc-83d8-6d8a0ab85e46 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.874147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb358d-2210-46bc-83d8-6d8a0ab85e46 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.874147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cb358d-2210-46bc-83d8-6d8a0ab85e46 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.874147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 56/107] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:08.874147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4167-53c8-41d9-b7d3-78a0dd81608f None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.874528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4167-53c8-41d9-b7d3-78a0dd81608f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.874528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaa4167-53c8-41d9-b7d3-78a0dd81608f None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.874528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 57/108] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:08.874726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af74897-7766-4d0c-bcbd-0544ccc7cddf None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:08.874798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0a81a-963a-4b1b-acd8-baa1028db8f4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.874798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0a81a-963a-4b1b-acd8-baa1028db8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.874798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f0a81a-963a-4b1b-acd8-baa1028db8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.894462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-44f0a81a-963a-4b1b-acd8-baa1028db8f4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:08.895654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-44f0a81a-963a-4b1b-acd8-baa1028db8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:08.950518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:08.981862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 58/109] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:08.981862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1b498-6ec4-4488-b131-02267f767d5f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:08.981862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1b498-6ec4-4488-b131-02267f767d5f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:08.981862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc1b498-6ec4-4488-b131-02267f767d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:08.983292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:08.994898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3dc1b498-6ec4-4488-b131-02267f767d5f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:08.996108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3dc1b498-6ec4-4488-b131-02267f767d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:09.005008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af74897-7766-4d0c-bcbd-0544ccc7cddf None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.040775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbd51b6-5348-4c70-a133-cd7df5e6e127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.041886 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd51b6-5348-4c70-a133-cd7df5e6e127 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.042203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd51b6-5348-4c70-a133-cd7df5e6e127 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.042453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbd51b6-5348-4c70-a133-cd7df5e6e127 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.043214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd51b6-5348-4c70-a133-cd7df5e6e127 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:09.043552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd51b6-5348-4c70-a133-cd7df5e6e127 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:09.044954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbd51b6-5348-4c70-a133-cd7df5e6e127 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:09.053335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 52/110] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:09.054592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033b37-0863-46a5-ba61-9c5c6053f0ca None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.054879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033b37-0863-46a5-ba61-9c5c6053f0ca None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.055185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d033b37-0863-46a5-ba61-9c5c6053f0ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.070190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 59/111] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:09.071043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1b498-6ec4-4488-b131-02267f767d5f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.083395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3d033b37-0863-46a5-ba61-9c5c6053f0ca None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:09.084942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3d033b37-0863-46a5-ba61-9c5c6053f0ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:09.155985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 53/112] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:09.156926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d033b37-0863-46a5-ba61-9c5c6053f0ca None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.196044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc1b498-6ec4-4488-b131-02267f767d5f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.222207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:09.234319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4032143-23cd-4d01-ad9f-a49f7fd1de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.235292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4032143-23cd-4d01-ad9f-a49f7fd1de9b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.235653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4032143-23cd-4d01-ad9f-a49f7fd1de9b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.235929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4032143-23cd-4d01-ad9f-a49f7fd1de9b None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.237113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4032143-23cd-4d01-ad9f-a49f7fd1de9b None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:09.238598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4032143-23cd-4d01-ad9f-a49f7fd1de9b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:09.240356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 60/113] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:09 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:09.241120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4032143-23cd-4d01-ad9f-a49f7fd1de9b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.243835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:09.270684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d033b37-0863-46a5-ba61-9c5c6053f0ca None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.305667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105d95-3bdc-46ab-96d9-08770dbf2d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.306557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105d95-3bdc-46ab-96d9-08770dbf2d0e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.306847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105d95-3bdc-46ab-96d9-08770dbf2d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.307176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2105d95-3bdc-46ab-96d9-08770dbf2d0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.318050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2105d95-3bdc-46ab-96d9-08770dbf2d0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-a2105d95-3bdc-46ab-96d9-08770dbf2d0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:09.325731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:09.326576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:09.327350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 54/114] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:09.328020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105d95-3bdc-46ab-96d9-08770dbf2d0e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.356824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4032143-23cd-4d01-ad9f-a49f7fd1de9b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.394239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742650aa-3349-49b2-85ad-c2efa61c6310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.395129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742650aa-3349-49b2-85ad-c2efa61c6310 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.395443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742650aa-3349-49b2-85ad-c2efa61c6310 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.395718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742650aa-3349-49b2-85ad-c2efa61c6310 None admin] PATH_INFO: `/v3/users/468703f0020b4014bb0b77a4767056b1` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.411768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742650aa-3349-49b2-85ad-c2efa61c6310 None admin] RBAC: Authorizing `identity:get_user(user_id=468703f0020b4014bb0b77a4767056b1)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:09.413309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742650aa-3349-49b2-85ad-c2efa61c6310 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:09.415340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 61/115] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:09 2022] GET /identity/v3/users/468703f0020b4014bb0b77a4767056b1 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:09.416533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742650aa-3349-49b2-85ad-c2efa61c6310 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.447033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2105d95-3bdc-46ab-96d9-08770dbf2d0e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.464287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:09.482109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969d7a4-4ae8-4857-bb4b-a9bf3bff71ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.482901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969d7a4-4ae8-4857-bb4b-a9bf3bff71ef None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.483214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969d7a4-4ae8-4857-bb4b-a9bf3bff71ef None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.483525 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c969d7a4-4ae8-4857-bb4b-a9bf3bff71ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.484686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969d7a4-4ae8-4857-bb4b-a9bf3bff71ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:09.486156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c969d7a4-4ae8-4857-bb4b-a9bf3bff71ef None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:09.487343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:09.487870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 55/116] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:09.493238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969d7a4-4ae8-4857-bb4b-a9bf3bff71ef None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.530511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742650aa-3349-49b2-85ad-c2efa61c6310 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.563193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9191f-cfea-4e32-85f4-f516826635b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.564038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9191f-cfea-4e32-85f4-f516826635b7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.564322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9191f-cfea-4e32-85f4-f516826635b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.564594 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9191f-cfea-4e32-85f4-f516826635b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.575477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9191f-cfea-4e32-85f4-f516826635b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-06f9191f-cfea-4e32-85f4-f516826635b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:09.583380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:09.584370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 62/117] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:09.590052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9191f-cfea-4e32-85f4-f516826635b7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.605802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c969d7a4-4ae8-4857-bb4b-a9bf3bff71ef None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.638129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4cf35-7b10-42db-8e32-364c610c50c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.638956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4cf35-7b10-42db-8e32-364c610c50c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.639239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4cf35-7b10-42db-8e32-364c610c50c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.639548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf4cf35-7b10-42db-8e32-364c610c50c8 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/users/ce1d4da72fcf4acabf4a7749465fe917/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.641519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4cf35-7b10-42db-8e32-364c610c50c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d512da984d845ada1d62c6bfc2ef6c7, user_id=ce1d4da72fcf4acabf4a7749465fe917, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:09.642988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf4cf35-7b10-42db-8e32-364c610c50c8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:09.656456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 56/118] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:09 2022] PUT /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/users/ce1d4da72fcf4acabf4a7749465fe917/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:09.657260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4cf35-7b10-42db-8e32-364c610c50c8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.697528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9191f-cfea-4e32-85f4-f516826635b7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.710328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:09.724865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:09.742734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.743489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.743781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.744065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.744866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:09.745181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:09.746635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:09.757446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 63/119] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:09.758107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.784411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf4cf35-7b10-42db-8e32-364c610c50c8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.821045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb901c2-25c1-4529-9403-05cc5c9cee8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.822011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb901c2-25c1-4529-9403-05cc5c9cee8c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.822460 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb901c2-25c1-4529-9403-05cc5c9cee8c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.822838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb901c2-25c1-4529-9403-05cc5c9cee8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.869382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6780718f-3239-447d-ae30-aecc638eddcb None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:09.897412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb901c2-25c1-4529-9403-05cc5c9cee8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:09.898805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb901c2-25c1-4529-9403-05cc5c9cee8c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:09.903841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ca99-c477-4ae1-91b2-e7333474dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:09.904521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ca99-c477-4ae1-91b2-e7333474dc0d None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:09.904805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ca99-c477-4ae1-91b2-e7333474dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:09.905084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a80ca99-c477-4ae1-91b2-e7333474dc0d None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:09.905800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80ca99-c477-4ae1-91b2-e7333474dc0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:09.907387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a80ca99-c477-4ae1-91b2-e7333474dc0d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:09.943288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 64/120] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 08:59:09 2022] POST /identity/v3/projects => generated 302 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:09.943288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ca99-c477-4ae1-91b2-e7333474dc0d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:09.952262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:09.982253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:10.035491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 57/121] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:10.037110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465574-2521-473e-93bb-607c46a8b609 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.037406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465574-2521-473e-93bb-607c46a8b609 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.037680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4465574-2521-473e-93bb-607c46a8b609 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.039218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 58/122] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:10.039860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4465574-2521-473e-93bb-607c46a8b609 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.068701 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a80ca99-c477-4ae1-91b2-e7333474dc0d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.120476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.122430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.123377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.124362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.125720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:10.126576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:10.129052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:10.147253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 65/123] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:10.149493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.175593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4465574-2521-473e-93bb-607c46a8b609 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.216183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b8d889-4cd4-47d6-878e-0bfe8549fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.217070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d889-4cd4-47d6-878e-0bfe8549fa65 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.217349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d889-4cd4-47d6-878e-0bfe8549fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.217619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b8d889-4cd4-47d6-878e-0bfe8549fa65 None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.218801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d889-4cd4-47d6-878e-0bfe8549fa65 None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:10.220302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b8d889-4cd4-47d6-878e-0bfe8549fa65 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:10.222107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 59/124] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:10 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:10.223058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd6b0-b204-4c18-b1a0-2a5ce6753941 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.224013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd6b0-b204-4c18-b1a0-2a5ce6753941 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.224297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd6b0-b204-4c18-b1a0-2a5ce6753941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.241463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3f7fd6b0-b204-4c18-b1a0-2a5ce6753941 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:10.242739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3f7fd6b0-b204-4c18-b1a0-2a5ce6753941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:10.256071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:10.261535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:10.267875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9283f96-deb6-4d0f-b910-78e7cd4a213b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.311938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff7d40-3d57-4c45-9d8e-54b4cbf26a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.313673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff7d40-3d57-4c45-9d8e-54b4cbf26a81 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.314438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff7d40-3d57-4c45-9d8e-54b4cbf26a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.314910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff7d40-3d57-4c45-9d8e-54b4cbf26a81 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.317215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff7d40-3d57-4c45-9d8e-54b4cbf26a81 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:10.319097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff7d40-3d57-4c45-9d8e-54b4cbf26a81 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:10.320890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 60/125] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:10.327307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 66/126] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:10 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:10.327448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fd6b0-b204-4c18-b1a0-2a5ce6753941 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.328193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff7d40-3d57-4c45-9d8e-54b4cbf26a81 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.460986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fd6b0-b204-4c18-b1a0-2a5ce6753941 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.467163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff7d40-3d57-4c45-9d8e-54b4cbf26a81 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.504621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453febdf-42fd-4a1b-9bdd-5e12ab5d1f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.505506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453febdf-42fd-4a1b-9bdd-5e12ab5d1f7e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.505788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453febdf-42fd-4a1b-9bdd-5e12ab5d1f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.506061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453febdf-42fd-4a1b-9bdd-5e12ab5d1f7e None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.507397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453febdf-42fd-4a1b-9bdd-5e12ab5d1f7e None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:10.509033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453febdf-42fd-4a1b-9bdd-5e12ab5d1f7e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:10.514738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9587e9-59dc-46b7-8f02-56c1f0195ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.515276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 61/127] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:10 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:10.516017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453febdf-42fd-4a1b-9bdd-5e12ab5d1f7e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.516688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9587e9-59dc-46b7-8f02-56c1f0195ceb None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.517215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9587e9-59dc-46b7-8f02-56c1f0195ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.517689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9587e9-59dc-46b7-8f02-56c1f0195ceb None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.520442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9587e9-59dc-46b7-8f02-56c1f0195ceb None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:10.522136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9587e9-59dc-46b7-8f02-56c1f0195ceb None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:10.524240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 67/128] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:10 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:10.529944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9587e9-59dc-46b7-8f02-56c1f0195ceb None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.535912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:10.575184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:10.630747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453febdf-42fd-4a1b-9bdd-5e12ab5d1f7e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.656224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9587e9-59dc-46b7-8f02-56c1f0195ceb None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.666283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.667159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.667325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.667632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.668473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] RBAC: Adding query filter params (role.id=dd10b97be2224165a8d312ca6eefdbfd, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7, user.id=468703f0020b4014bb0b77a4767056b1) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:10.668784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:10.670228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:10.676302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 62/129] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:10 2022] GET /identity/v3/role_assignments?user.id=468703f0020b4014bb0b77a4767056b1&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=dd10b97be2224165a8d312ca6eefdbfd => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:10.677161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.706382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19481c-800b-4e01-a2b0-fa4df422e862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.708379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19481c-800b-4e01-a2b0-fa4df422e862 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.709055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19481c-800b-4e01-a2b0-fa4df422e862 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.709677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19481c-800b-4e01-a2b0-fa4df422e862 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.725643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19481c-800b-4e01-a2b0-fa4df422e862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-da19481c-800b-4e01-a2b0-fa4df422e862 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:10.737275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:10.738252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:10.739056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:10.740799 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 68/130] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:10.746202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19481c-800b-4e01-a2b0-fa4df422e862 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.811160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:10.840717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc1fe8-acce-47b7-8331-b5562f2896fb None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.885991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575e534-89dc-4344-8457-3759e5cf146e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.887378 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575e534-89dc-4344-8457-3759e5cf146e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.887893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575e534-89dc-4344-8457-3759e5cf146e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.888384 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575e534-89dc-4344-8457-3759e5cf146e None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.889890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575e534-89dc-4344-8457-3759e5cf146e None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:10.891813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b575e534-89dc-4344-8457-3759e5cf146e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:10.895416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 63/131] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:10 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:10.896498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575e534-89dc-4344-8457-3759e5cf146e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:10.929776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:10.939956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19481c-800b-4e01-a2b0-fa4df422e862 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:10.976276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae01d8-0423-4d16-8aef-c6be396ee68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:10.977201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae01d8-0423-4d16-8aef-c6be396ee68c None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:10.977516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae01d8-0423-4d16-8aef-c6be396ee68c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:10.977804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeae01d8-0423-4d16-8aef-c6be396ee68c None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/users/bdd8156d03f1489f99984f234d7d8ca7/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:10.979957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae01d8-0423-4d16-8aef-c6be396ee68c None admin] RBAC: Authorizing `identity:create_grant(project_id=0d512da984d845ada1d62c6bfc2ef6c7, user_id=bdd8156d03f1489f99984f234d7d8ca7, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:10.981574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeae01d8-0423-4d16-8aef-c6be396ee68c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:11.014840 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 69/132] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:10 2022] PUT /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/users/bdd8156d03f1489f99984f234d7d8ca7/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:11.015706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae01d8-0423-4d16-8aef-c6be396ee68c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:11.028477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b575e534-89dc-4344-8457-3759e5cf146e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:11.071059 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:11.072260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:11.073041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:11.073519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:11.074695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:11.075231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:11.077309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:11.089686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 64/133] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:11.096278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:11.107782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:11.219310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537178-0923-4f7d-b70a-600b129968e0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:11.260252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c0b2e-7101-4388-9583-e91e70ff28fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:11.261503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c0b2e-7101-4388-9583-e91e70ff28fb None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:11.262006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c0b2e-7101-4388-9583-e91e70ff28fb None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:11.262486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8c0b2e-7101-4388-9583-e91e70ff28fb None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:11.263965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c0b2e-7101-4388-9583-e91e70ff28fb None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:11.267458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8c0b2e-7101-4388-9583-e91e70ff28fb None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:11.273240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeae01d8-0423-4d16-8aef-c6be396ee68c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:11.274418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 65/134] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:11 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:11.280075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c0b2e-7101-4388-9583-e91e70ff28fb None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:11.331739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9ac96-0dcc-4765-8cef-9234185012ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:11.333136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9ac96-0dcc-4765-8cef-9234185012ff None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:11.333629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9ac96-0dcc-4765-8cef-9234185012ff None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:11.334291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9ac96-0dcc-4765-8cef-9234185012ff None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/users/bdd8156d03f1489f99984f234d7d8ca7/roles/2e550a6a96bb4804926c6213a5af7585` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:11.337124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9ac96-0dcc-4765-8cef-9234185012ff None admin] RBAC: Authorizing `identity:create_grant(project_id=0d512da984d845ada1d62c6bfc2ef6c7, user_id=bdd8156d03f1489f99984f234d7d8ca7, role_id=2e550a6a96bb4804926c6213a5af7585)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:11.339026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c9ac96-0dcc-4765-8cef-9234185012ff None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:11.355765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 70/135] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:11 2022] PUT /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/users/bdd8156d03f1489f99984f234d7d8ca7/roles/2e550a6a96bb4804926c6213a5af7585 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:11.418209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:11.448314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8c0b2e-7101-4388-9583-e91e70ff28fb None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:11.488932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15217e67-af1b-4d6d-88fd-2bc05ab36ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:11.490326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e67-af1b-4d6d-88fd-2bc05ab36ee3 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:11.490846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e67-af1b-4d6d-88fd-2bc05ab36ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:11.491368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15217e67-af1b-4d6d-88fd-2bc05ab36ee3 None admin] PATH_INFO: `/v3/projects/a2ceac24453940eea8e39cedc308fd02` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:11.493606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15217e67-af1b-4d6d-88fd-2bc05ab36ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=a2ceac24453940eea8e39cedc308fd02)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:11.495644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15217e67-af1b-4d6d-88fd-2bc05ab36ee3 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:11.497783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 66/136] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:11 2022] GET /identity/v3/projects/a2ceac24453940eea8e39cedc308fd02 => generated 312 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:11.503801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9ac96-0dcc-4765-8cef-9234185012ff None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:11.525449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:11.627805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9ac96-0dcc-4765-8cef-9234185012ff None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:11.670330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b5340-caaa-4ea1-b522-a247dac68e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:11.671326 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b5340-caaa-4ea1-b522-a247dac68e3f None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:11.671558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b5340-caaa-4ea1-b522-a247dac68e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:11.671836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b5340-caaa-4ea1-b522-a247dac68e3f None admin] PATH_INFO: `/v3/projects/a2ceac24453940eea8e39cedc308fd02/users/bdd8156d03f1489f99984f234d7d8ca7/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:11.673848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b5340-caaa-4ea1-b522-a247dac68e3f None admin] RBAC: Authorizing `identity:create_grant(project_id=a2ceac24453940eea8e39cedc308fd02, user_id=bdd8156d03f1489f99984f234d7d8ca7, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:11.675347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14b5340-caaa-4ea1-b522-a247dac68e3f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:11.705440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 71/137] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:11 2022] PUT /identity/v3/projects/a2ceac24453940eea8e39cedc308fd02/users/bdd8156d03f1489f99984f234d7d8ca7/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:12.099733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf1293-3d59-4404-b9f7-c3b3fadb94a9 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:12.100108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf1293-3d59-4404-b9f7-c3b3fadb94a9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:12.100333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bf1293-3d59-4404-b9f7-c3b3fadb94a9 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:12.102462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 67/138] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:12.107839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb64be-42c1-460f-837a-9a6bc70720be None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:12.108375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb64be-42c1-460f-837a-9a6bc70720be None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:12.108861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2deb64be-42c1-460f-837a-9a6bc70720be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:12.127200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-2deb64be-42c1-460f-837a-9a6bc70720be None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:12.128733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2deb64be-42c1-460f-837a-9a6bc70720be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:12.202305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 72/139] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:12.231123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf1293-3d59-4404-b9f7-c3b3fadb94a9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:12.353920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bf1293-3d59-4404-b9f7-c3b3fadb94a9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:12.388808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c3867-748c-495b-9ddc-52eb2daf9ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:12.389621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c3867-748c-495b-9ddc-52eb2daf9ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:12.389914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c3867-748c-495b-9ddc-52eb2daf9ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:12.390195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2c3867-748c-495b-9ddc-52eb2daf9ed6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:12.401470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2c3867-748c-495b-9ddc-52eb2daf9ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-2d2c3867-748c-495b-9ddc-52eb2daf9ed6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:12.409304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:12.410146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:12.410926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 68/140] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:12.418342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb64be-42c1-460f-837a-9a6bc70720be None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:12.421248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:12.540804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2deb64be-42c1-460f-837a-9a6bc70720be None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:12.580266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:12.581373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:12.581844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:12.582345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:12.583352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:12.583862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:12.585435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:12.596508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 73/141] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:12.602459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c3867-748c-495b-9ddc-52eb2daf9ed6 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:12.610309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:12.720870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2c3867-748c-495b-9ddc-52eb2daf9ed6 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:12.758845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662f42b-e89e-49e2-9b0f-0c74a9df6c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:12.759758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662f42b-e89e-49e2-9b0f-0c74a9df6c8b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:12.760040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662f42b-e89e-49e2-9b0f-0c74a9df6c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:12.760320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662f42b-e89e-49e2-9b0f-0c74a9df6c8b None admin] PATH_INFO: `/v3/users/ce1d4da72fcf4acabf4a7749465fe917` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:12.763972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662f42b-e89e-49e2-9b0f-0c74a9df6c8b None admin] RBAC: Authorizing `identity:get_user(user_id=ce1d4da72fcf4acabf4a7749465fe917)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:12.765907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9662f42b-e89e-49e2-9b0f-0c74a9df6c8b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:12.767889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 69/142] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:12 2022] GET /identity/v3/users/ce1d4da72fcf4acabf4a7749465fe917 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:12.774382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:12.882987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:12.939120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32259ecc-be9f-429b-ac12-ad19091347ff None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:12.989570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af5cf-7ad8-4b63-9bbc-572d9af6e612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:12.990356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af5cf-7ad8-4b63-9bbc-572d9af6e612 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:12.990646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af5cf-7ad8-4b63-9bbc-572d9af6e612 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:12.990923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00af5cf-7ad8-4b63-9bbc-572d9af6e612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:13.078586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00af5cf-7ad8-4b63-9bbc-572d9af6e612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:13.080121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00af5cf-7ad8-4b63-9bbc-572d9af6e612 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:13.146789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:13.183580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d0927-7902-4a10-9672-4cb715e796ef None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:13.183919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d0927-7902-4a10-9672-4cb715e796ef None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:13.184145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d0927-7902-4a10-9672-4cb715e796ef None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:13.185754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 70/143] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:13.197401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf84e-0afa-4150-ae98-2206304e1d69 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:13.197689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf84e-0afa-4150-ae98-2206304e1d69 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:13.197960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdf84e-0afa-4150-ae98-2206304e1d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:13.222383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 74/144] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:13.228414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-98bdf84e-0afa-4150-ae98-2206304e1d69 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:13.228549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af5cf-7ad8-4b63-9bbc-572d9af6e612 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:13.229608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-98bdf84e-0afa-4150-ae98-2206304e1d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:13.302269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:13.311540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 71/145] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:13.383286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00af5cf-7ad8-4b63-9bbc-572d9af6e612 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:13.390236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdf84e-0afa-4150-ae98-2206304e1d69 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:13.465537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fca6a-5798-47d0-a6f5-98db2cf55f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:13.466341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fca6a-5798-47d0-a6f5-98db2cf55f0c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:13.466626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fca6a-5798-47d0-a6f5-98db2cf55f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:13.466901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684fca6a-5798-47d0-a6f5-98db2cf55f0c None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:13.468219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fca6a-5798-47d0-a6f5-98db2cf55f0c None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:13.469700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684fca6a-5798-47d0-a6f5-98db2cf55f0c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:13.471606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 75/146] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:13 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:13.478341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fca6a-5798-47d0-a6f5-98db2cf55f0c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:13.542165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdf84e-0afa-4150-ae98-2206304e1d69 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:13.571665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:13.581817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfee764-7c02-4e2e-b408-591101c5d05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:13.583509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee764-7c02-4e2e-b408-591101c5d05a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:13.583801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee764-7c02-4e2e-b408-591101c5d05a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:13.584080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfee764-7c02-4e2e-b408-591101c5d05a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:13.585312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee764-7c02-4e2e-b408-591101c5d05a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:13.586825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfee764-7c02-4e2e-b408-591101c5d05a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:13.588867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 72/147] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:13.599642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfee764-7c02-4e2e-b408-591101c5d05a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:13.692765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684fca6a-5798-47d0-a6f5-98db2cf55f0c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:13.719408 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:13.727358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf99a879-9ed1-4703-982c-840c218d44d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:13.728293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99a879-9ed1-4703-982c-840c218d44d3 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:13.728572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99a879-9ed1-4703-982c-840c218d44d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:13.728848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf99a879-9ed1-4703-982c-840c218d44d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:13.729690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99a879-9ed1-4703-982c-840c218d44d3 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7, user.id=ce1d4da72fcf4acabf4a7749465fe917) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:13.730002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99a879-9ed1-4703-982c-840c218d44d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:13.731507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf99a879-9ed1-4703-982c-840c218d44d3 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:13.737821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 76/148] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:13 2022] GET /identity/v3/role_assignments?user.id=ce1d4da72fcf4acabf4a7749465fe917&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=6dca675ade814980a252f800fb4fa3ad => generated 608 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:13.770763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfee764-7c02-4e2e-b408-591101c5d05a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:13.815033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561ce0e3-455d-4207-b8b1-6dfbd4306344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:13.816012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ce0e3-455d-4207-b8b1-6dfbd4306344 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:13.816439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ce0e3-455d-4207-b8b1-6dfbd4306344 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:13.816855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561ce0e3-455d-4207-b8b1-6dfbd4306344 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:13.817784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ce0e3-455d-4207-b8b1-6dfbd4306344 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:13.819498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561ce0e3-455d-4207-b8b1-6dfbd4306344 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:13.859794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 73/149] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:13 2022] POST /identity/v3/users => generated 283 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:14.195117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab877cf-35ed-4f52-9147-95c92cfaaf46 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:14.195428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab877cf-35ed-4f52-9147-95c92cfaaf46 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:14.195761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab877cf-35ed-4f52-9147-95c92cfaaf46 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:14.197148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 77/150] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:14.202815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ffead-90fa-41e5-b452-49f293833c83 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:14.203111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ffead-90fa-41e5-b452-49f293833c83 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:14.203429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ffead-90fa-41e5-b452-49f293833c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:14.226093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5d6ffead-90fa-41e5-b452-49f293833c83 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:14.227676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5d6ffead-90fa-41e5-b452-49f293833c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:14.307775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 74/151] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:14.340420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab877cf-35ed-4f52-9147-95c92cfaaf46 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:14.471590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:14.478207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab877cf-35ed-4f52-9147-95c92cfaaf46 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:14.518170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca7b1d-f6d3-481d-bd11-be8d83fcf39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:14.518985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca7b1d-f6d3-481d-bd11-be8d83fcf39b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:14.519308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca7b1d-f6d3-481d-bd11-be8d83fcf39b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:14.519565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca7b1d-f6d3-481d-bd11-be8d83fcf39b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:14.530052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca7b1d-f6d3-481d-bd11-be8d83fcf39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-deca7b1d-f6d3-481d-bd11-be8d83fcf39b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:14.537894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:14.538734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:14.540302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 78/152] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:14.545343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ffead-90fa-41e5-b452-49f293833c83 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:14.607412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:14.639645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043babe1-dc68-4bab-b517-94164047c765 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:14.639941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043babe1-dc68-4bab-b517-94164047c765 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:14.640209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043babe1-dc68-4bab-b517-94164047c765 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:14.641597 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 79/153] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:14.647807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caffd0-4e1e-4e30-b198-1bf22aa47255 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:14.648142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caffd0-4e1e-4e30-b198-1bf22aa47255 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:14.648373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7caffd0-4e1e-4e30-b198-1bf22aa47255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:14.669566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ffead-90fa-41e5-b452-49f293833c83 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:14.679065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b7caffd0-4e1e-4e30-b198-1bf22aa47255 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:14.683478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b7caffd0-4e1e-4e30-b198-1bf22aa47255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:14.720838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:14.721881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:14.722299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:14.722955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:14.724006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:14.724363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:14.725771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:14.737054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 75/154] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:14.743306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:14.820069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 80/155] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:14.857640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c9f21a-fe16-4565-9dd3-c4e94675a59e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:14.858005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caffd0-4e1e-4e30-b198-1bf22aa47255 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:14.878697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:14.902769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15399d5c-aa5b-4a2d-80ee-d18c1305fe1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:14.903926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15399d5c-aa5b-4a2d-80ee-d18c1305fe1c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:14.904370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15399d5c-aa5b-4a2d-80ee-d18c1305fe1c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:14.904843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15399d5c-aa5b-4a2d-80ee-d18c1305fe1c None admin] PATH_INFO: `/v3/users/468703f0020b4014bb0b77a4767056b1` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:14.906208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15399d5c-aa5b-4a2d-80ee-d18c1305fe1c None admin] RBAC: Authorizing `identity:get_user(user_id=468703f0020b4014bb0b77a4767056b1)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:14.908120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15399d5c-aa5b-4a2d-80ee-d18c1305fe1c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:14.910330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 76/156] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:14 2022] GET /identity/v3/users/468703f0020b4014bb0b77a4767056b1 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:14.912110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91d7e-8c9e-48f1-9e9a-0962ab7de6ae None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:14.912395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91d7e-8c9e-48f1-9e9a-0962ab7de6ae None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:14.912673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91d7e-8c9e-48f1-9e9a-0962ab7de6ae None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:14.918092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 77/157] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:14.918901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe91d7e-8c9e-48f1-9e9a-0962ab7de6ae None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:14.974729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:14.994735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7caffd0-4e1e-4e30-b198-1bf22aa47255 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.045793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e57e5-dde5-40f8-8d72-f784020325f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.047334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e57e5-dde5-40f8-8d72-f784020325f9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.047796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e57e5-dde5-40f8-8d72-f784020325f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.048269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e57e5-dde5-40f8-8d72-f784020325f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.050860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe91d7e-8c9e-48f1-9e9a-0962ab7de6ae None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.064228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e57e5-dde5-40f8-8d72-f784020325f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-903e57e5-dde5-40f8-8d72-f784020325f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:15.074899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:15.075938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:15.076711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:15.077347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 81/158] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:15.078430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f064b-7be6-4348-8adf-1af4faf75c6a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.078793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f064b-7be6-4348-8adf-1af4faf75c6a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.078793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738f064b-7be6-4348-8adf-1af4faf75c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.095357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5a021-7fae-4d1f-8cba-8488307212f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.096237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5a021-7fae-4d1f-8cba-8488307212f9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.096514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5a021-7fae-4d1f-8cba-8488307212f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.096782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a5a021-7fae-4d1f-8cba-8488307212f9 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.098877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5a021-7fae-4d1f-8cba-8488307212f9 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:15.099698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-738f064b-7be6-4348-8adf-1af4faf75c6a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:15.100374 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a5a021-7fae-4d1f-8cba-8488307212f9 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:15.101307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-738f064b-7be6-4348-8adf-1af4faf75c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:15.102180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 78/159] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:14 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:15.103036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5a021-7fae-4d1f-8cba-8488307212f9 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:15.163642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:15.179642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 82/160] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:15.184654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5a9ea-a097-4950-8f64-8358b803751c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.184943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5a9ea-a097-4950-8f64-8358b803751c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.185276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5a9ea-a097-4950-8f64-8358b803751c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.186662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 83/161] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:15.187479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5a9ea-a097-4950-8f64-8358b803751c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:15.212717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a5a021-7fae-4d1f-8cba-8488307212f9 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.251077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b7762-75b4-4050-a62d-fc5ce7ef744f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.251692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b7762-75b4-4050-a62d-fc5ce7ef744f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.251972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b7762-75b4-4050-a62d-fc5ce7ef744f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.252364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b7762-75b4-4050-a62d-fc5ce7ef744f None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.253124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b7762-75b4-4050-a62d-fc5ce7ef744f None admin] RBAC: Adding query filter params (name=member) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:15.253426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b7762-75b4-4050-a62d-fc5ce7ef744f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:15.254932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b7762-75b4-4050-a62d-fc5ce7ef744f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:15.264637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 79/162] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:15.266373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b83b-c3cb-4e40-940c-0d64e55b78e1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.266800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b83b-c3cb-4e40-940c-0d64e55b78e1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.267238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff0b83b-c3cb-4e40-940c-0d64e55b78e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.285411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-fff0b83b-c3cb-4e40-940c-0d64e55b78e1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:15.287972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-fff0b83b-c3cb-4e40-940c-0d64e55b78e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:15.297284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:15.325273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5a9ea-a097-4950-8f64-8358b803751c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.361795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 80/163] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:15.362820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0b83b-c3cb-4e40-940c-0d64e55b78e1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:15.363140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbfade-296e-4d44-a02b-4de9e13c8939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.364235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfade-296e-4d44-a02b-4de9e13c8939 None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.364670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfade-296e-4d44-a02b-4de9e13c8939 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.365241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbbfade-296e-4d44-a02b-4de9e13c8939 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/users/468703f0020b4014bb0b77a4767056b1/roles/dd10b97be2224165a8d312ca6eefdbfd` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.367622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbfade-296e-4d44-a02b-4de9e13c8939 None admin] RBAC: Authorizing `identity:create_grant(project_id=0d512da984d845ada1d62c6bfc2ef6c7, user_id=468703f0020b4014bb0b77a4767056b1, role_id=dd10b97be2224165a8d312ca6eefdbfd)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:15.369269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbbfade-296e-4d44-a02b-4de9e13c8939 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:15.397921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 84/164] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:15 2022] PUT /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/users/468703f0020b4014bb0b77a4767056b1/roles/dd10b97be2224165a8d312ca6eefdbfd => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:15.399297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbfade-296e-4d44-a02b-4de9e13c8939 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:15.416187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:15.473192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff0b83b-c3cb-4e40-940c-0d64e55b78e1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.514826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4bea3-0b02-43ed-bc44-536f35dd648a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.515866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4bea3-0b02-43ed-bc44-536f35dd648a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.516245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4bea3-0b02-43ed-bc44-536f35dd648a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.516640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4bea3-0b02-43ed-bc44-536f35dd648a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.528685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4bea3-0b02-43ed-bc44-536f35dd648a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-c3c4bea3-0b02-43ed-bc44-536f35dd648a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:15.537046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:15.537994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:15.538759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 81/165] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:15.539462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4bea3-0b02-43ed-bc44-536f35dd648a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:15.575189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbbfade-296e-4d44-a02b-4de9e13c8939 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.648712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902b49e-5221-4b38-bb3d-bf5a791445a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.649712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902b49e-5221-4b38-bb3d-bf5a791445a4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.650044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902b49e-5221-4b38-bb3d-bf5a791445a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.650327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902b49e-5221-4b38-bb3d-bf5a791445a4 None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.651717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902b49e-5221-4b38-bb3d-bf5a791445a4 None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:15.653397 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902b49e-5221-4b38-bb3d-bf5a791445a4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:15.654978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4bea3-0b02-43ed-bc44-536f35dd648a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.655419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 85/166] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:15 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:15.656307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902b49e-5221-4b38-bb3d-bf5a791445a4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:15.659838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:15.673899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:15.705597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50306fdc-595a-4d7a-a89c-72ce94925db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.706729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50306fdc-595a-4d7a-a89c-72ce94925db2 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.707246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50306fdc-595a-4d7a-a89c-72ce94925db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.707743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50306fdc-595a-4d7a-a89c-72ce94925db2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.721739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50306fdc-595a-4d7a-a89c-72ce94925db2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-50306fdc-595a-4d7a-a89c-72ce94925db2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:15.732087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:15.733007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:15.733825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:15.736163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 82/167] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:15.737150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50306fdc-595a-4d7a-a89c-72ce94925db2 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:15.806166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902b49e-5221-4b38-bb3d-bf5a791445a4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.891966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.892656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.892946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.893213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:15.893984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:15.894295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:15.895730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:15.909684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 86/168] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:15.910500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:15.927830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50306fdc-595a-4d7a-a89c-72ce94925db2 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:15.965803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290de8c-7623-46d8-bb0f-78c30a91adee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:15.966891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290de8c-7623-46d8-bb0f-78c30a91adee None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:15.967396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290de8c-7623-46d8-bb0f-78c30a91adee None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:15.967938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290de8c-7623-46d8-bb0f-78c30a91adee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:16.001887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:16.042884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290de8c-7623-46d8-bb0f-78c30a91adee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:16.044882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290de8c-7623-46d8-bb0f-78c30a91adee None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:16.055848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:16.154134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 83/169] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:16.159386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cd919-d00c-4988-9e4a-7a4d8301d828 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:16.159782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290de8c-7623-46d8-bb0f-78c30a91adee None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:16.229107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:16.266047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290de8c-7623-46d8-bb0f-78c30a91adee None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:16.290558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:16.291707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:16.292007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:16.292513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:16.293080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:16.293324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:16.294725 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:16.304867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c316-e1b4-468e-a4f2-377386e9e217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:16.306268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c316-e1b4-468e-a4f2-377386e9e217 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:16.306750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c316-e1b4-468e-a4f2-377386e9e217 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:16.307220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40c316-e1b4-468e-a4f2-377386e9e217 None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:16.308758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40c316-e1b4-468e-a4f2-377386e9e217 None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:16.310505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40c316-e1b4-468e-a4f2-377386e9e217 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:16.312833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 84/170] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:16 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:16.313438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 87/171] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:16.314253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c316-e1b4-468e-a4f2-377386e9e217 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:16.320258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:16.436862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40c316-e1b4-468e-a4f2-377386e9e217 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:16.476169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3e939-6de0-49e1-9425-67069784b640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:16.477490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e939-6de0-49e1-9425-67069784b640 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:16.477972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e939-6de0-49e1-9425-67069784b640 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:16.478439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3e939-6de0-49e1-9425-67069784b640 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:16.480476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3e939-6de0-49e1-9425-67069784b640 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:16.482184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3e939-6de0-49e1-9425-67069784b640 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:16.484516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 85/172] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:16 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:16.487287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:16.488470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3e939-6de0-49e1-9425-67069784b640 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:16.558744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fede982-13ed-4c0a-b8cf-64bece19f54d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:16.596676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:16.623981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728111-70ad-45d9-a2f4-383575349a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:16.624656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728111-70ad-45d9-a2f4-383575349a10 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:16.624938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728111-70ad-45d9-a2f4-383575349a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:16.625214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb728111-70ad-45d9-a2f4-383575349a10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:16.653407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3e939-6de0-49e1-9425-67069784b640 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:16.708068 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3596b-5c19-4a33-ab5a-e68fe5f71b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:16.709009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3596b-5c19-4a33-ab5a-e68fe5f71b9c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:16.709289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3596b-5c19-4a33-ab5a-e68fe5f71b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:16.709563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3596b-5c19-4a33-ab5a-e68fe5f71b9c None admin] PATH_INFO: `/v3/users/bdd8156d03f1489f99984f234d7d8ca7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:16.711044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3596b-5c19-4a33-ab5a-e68fe5f71b9c None admin] RBAC: Authorizing `identity:get_user(user_id=bdd8156d03f1489f99984f234d7d8ca7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:16.712564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3596b-5c19-4a33-ab5a-e68fe5f71b9c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:16.714210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 86/173] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:16 2022] GET /identity/v3/users/bdd8156d03f1489f99984f234d7d8ca7 => generated 275 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:16.715028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3596b-5c19-4a33-ab5a-e68fe5f71b9c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:16.734639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb728111-70ad-45d9-a2f4-383575349a10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:16.744268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb728111-70ad-45d9-a2f4-383575349a10 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:16.852419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:16.888271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3596b-5c19-4a33-ab5a-e68fe5f71b9c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:16.923801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:16.924751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:16.925233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:16.925675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:16.926681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] RBAC: Adding query filter params (role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7, user.id=bdd8156d03f1489f99984f234d7d8ca7) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:16.927161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:16.928809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:16.935543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 87/174] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:16 2022] GET /identity/v3/role_assignments?user.id=bdd8156d03f1489f99984f234d7d8ca7&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 608 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:16.936532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:16.937633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 88/175] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:16.943336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728111-70ad-45d9-a2f4-383575349a10 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:17.012455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:17.046769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8647aa-6e4b-42c9-91e8-670789beab37 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:17.079085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:17.090400 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b3883-d183-4bbc-80cc-83f421cbb033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:17.091465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3883-d183-4bbc-80cc-83f421cbb033 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:17.091942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3883-d183-4bbc-80cc-83f421cbb033 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:17.092432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0b3883-d183-4bbc-80cc-83f421cbb033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:17.178008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb728111-70ad-45d9-a2f4-383575349a10 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:17.208752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b3883-d183-4bbc-80cc-83f421cbb033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:17.210357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0b3883-d183-4bbc-80cc-83f421cbb033 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:17.248342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939d526-7211-4e88-aa29-4cf362418ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:17.249211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939d526-7211-4e88-aa29-4cf362418ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:17.249486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939d526-7211-4e88-aa29-4cf362418ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:17.249756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939d526-7211-4e88-aa29-4cf362418ee7 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:17.252804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939d526-7211-4e88-aa29-4cf362418ee7 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:17.254265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939d526-7211-4e88-aa29-4cf362418ee7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:17.256120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 89/176] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:16 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:17.263040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939d526-7211-4e88-aa29-4cf362418ee7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:17.353181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 88/177] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:17.359179 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b3883-d183-4bbc-80cc-83f421cbb033 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:17.397932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:17.457374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:17.465424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939d526-7211-4e88-aa29-4cf362418ee7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:17.513661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0b3883-d183-4bbc-80cc-83f421cbb033 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:17.532451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:17.533158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:17.533439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:17.533760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:17.534621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] RBAC: Adding query filter params (role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7, user.id=bdd8156d03f1489f99984f234d7d8ca7) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:17.534931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:17.536436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:17.546748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 90/178] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:17 2022] GET /identity/v3/role_assignments?user.id=bdd8156d03f1489f99984f234d7d8ca7&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:17.566399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd688f5-664f-441d-9035-0e509cfad954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:17.567307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd688f5-664f-441d-9035-0e509cfad954 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:17.567606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd688f5-664f-441d-9035-0e509cfad954 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:17.567884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd688f5-664f-441d-9035-0e509cfad954 None admin] PATH_INFO: `/v3/projects/a2ceac24453940eea8e39cedc308fd02` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:17.569122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd688f5-664f-441d-9035-0e509cfad954 None admin] RBAC: Authorizing `identity:get_project(project_id=a2ceac24453940eea8e39cedc308fd02)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:17.570616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd688f5-664f-441d-9035-0e509cfad954 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:17.572366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 89/179] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:17 2022] GET /identity/v3/projects/a2ceac24453940eea8e39cedc308fd02 => generated 312 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:17.578809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:17.725073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4561f3f-89e2-4afe-9952-63de50deb781 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:17.773851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7376a691-20ea-452e-a60c-92fcae809690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:17.774589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376a691-20ea-452e-a60c-92fcae809690 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:17.774878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376a691-20ea-452e-a60c-92fcae809690 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:17.775152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7376a691-20ea-452e-a60c-92fcae809690 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:17.776571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376a691-20ea-452e-a60c-92fcae809690 None admin] RBAC: Adding query filter params (role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=a2ceac24453940eea8e39cedc308fd02, user.id=bdd8156d03f1489f99984f234d7d8ca7) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:17.776571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376a691-20ea-452e-a60c-92fcae809690 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:17.783288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7376a691-20ea-452e-a60c-92fcae809690 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:17.785468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 91/180] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:17 2022] GET /identity/v3/role_assignments?user.id=bdd8156d03f1489f99984f234d7d8ca7&scope.project.id=a2ceac24453940eea8e39cedc308fd02&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:17.797481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046c0a-d614-4327-af8d-abf843d70ea4 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:17.797777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046c0a-d614-4327-af8d-abf843d70ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:17.798034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4046c0a-d614-4327-af8d-abf843d70ea4 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:17.799718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 90/181] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:17.805519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254dcf7-8d0c-4a0e-aaaa-9b18559cf1b8 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:17.805808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254dcf7-8d0c-4a0e-aaaa-9b18559cf1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:17.806095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7254dcf7-8d0c-4a0e-aaaa-9b18559cf1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:17.827515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-7254dcf7-8d0c-4a0e-aaaa-9b18559cf1b8 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:17.828755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7254dcf7-8d0c-4a0e-aaaa-9b18559cf1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:17.849437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:17.928956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 92/182] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:17.965179 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4046c0a-d614-4327-af8d-abf843d70ea4 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:18.075005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:18.080098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4046c0a-d614-4327-af8d-abf843d70ea4 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:18.116393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6968442-b7a1-4023-8005-d08d61cbfee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:18.117189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6968442-b7a1-4023-8005-d08d61cbfee3 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.117499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6968442-b7a1-4023-8005-d08d61cbfee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.118864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6968442-b7a1-4023-8005-d08d61cbfee3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.128754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6968442-b7a1-4023-8005-d08d61cbfee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-d6968442-b7a1-4023-8005-d08d61cbfee3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:18.136664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:18.137597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:18.139823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 91/183] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:18.147282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254dcf7-8d0c-4a0e-aaaa-9b18559cf1b8 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:18.264813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7254dcf7-8d0c-4a0e-aaaa-9b18559cf1b8 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:18.313651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50fa3e-df95-4c69-abfa-5c01c8e9d503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:18.314391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50fa3e-df95-4c69-abfa-5c01c8e9d503 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.314676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50fa3e-df95-4c69-abfa-5c01c8e9d503 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.314987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50fa3e-df95-4c69-abfa-5c01c8e9d503 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.315818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50fa3e-df95-4c69-abfa-5c01c8e9d503 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:18.316127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50fa3e-df95-4c69-abfa-5c01c8e9d503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:18.317556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50fa3e-df95-4c69-abfa-5c01c8e9d503 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:18.329227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 93/184] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:18.334689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6968442-b7a1-4023-8005-d08d61cbfee3 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:18.458240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6968442-b7a1-4023-8005-d08d61cbfee3 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:18.482638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986a3a-2473-41d4-a0d4-6f3fdd26e37a None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.482950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986a3a-2473-41d4-a0d4-6f3fdd26e37a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.483454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3986a3a-2473-41d4-a0d4-6f3fdd26e37a None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.485748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 94/185] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:18.498127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c6a8-21b6-4ab5-bea0-a759ea6641b5 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.498430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c6a8-21b6-4ab5-bea0-a759ea6641b5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.498841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088c6a8-21b6-4ab5-bea0-a759ea6641b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.500580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426e07a6-2bc6-46f8-b1ab-0eb4fbb938d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:18.501454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e07a6-2bc6-46f8-b1ab-0eb4fbb938d8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.501705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e07a6-2bc6-46f8-b1ab-0eb4fbb938d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.501992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426e07a6-2bc6-46f8-b1ab-0eb4fbb938d8 None admin] PATH_INFO: `/v3/users/cf16d55421fb44f99dedfb8d0d28ced6` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.517197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a088c6a8-21b6-4ab5-bea0-a759ea6641b5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:18.518416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a088c6a8-21b6-4ab5-bea0-a759ea6641b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:18.519618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e07a6-2bc6-46f8-b1ab-0eb4fbb938d8 None admin] RBAC: Authorizing `identity:get_user(user_id=cf16d55421fb44f99dedfb8d0d28ced6)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:18.522003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426e07a6-2bc6-46f8-b1ab-0eb4fbb938d8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:18.524127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 92/186] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:18 2022] GET /identity/v3/users/cf16d55421fb44f99dedfb8d0d28ced6 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:18.525235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307aa1-23d0-4d70-ab9b-65f0d5a26299 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.525513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307aa1-23d0-4d70-ab9b-65f0d5a26299 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.525784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa307aa1-23d0-4d70-ab9b-65f0d5a26299 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.527509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 93/187] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:18.531421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307aa1-23d0-4d70-ab9b-65f0d5a26299 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:18.534442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:18.552036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:18.598623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 95/188] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:18.599216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45b2e4-7424-4842-8506-f673d94ce291 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.599530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45b2e4-7424-4842-8506-f673d94ce291 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.599808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd45b2e4-7424-4842-8506-f673d94ce291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.617039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-dd45b2e4-7424-4842-8506-f673d94ce291 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:18.617900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-dd45b2e4-7424-4842-8506-f673d94ce291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:18.666972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa307aa1-23d0-4d70-ab9b-65f0d5a26299 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:18.702707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 96/189] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:18.704740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45b2e4-7424-4842-8506-f673d94ce291 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:18.709251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50c8fc-6e18-4ac6-a680-092e77d745f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:18.710666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50c8fc-6e18-4ac6-a680-092e77d745f2 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.710666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50c8fc-6e18-4ac6-a680-092e77d745f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.710666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50c8fc-6e18-4ac6-a680-092e77d745f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.785698 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50c8fc-6e18-4ac6-a680-092e77d745f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:18.787181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50c8fc-6e18-4ac6-a680-092e77d745f2 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:18.794425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:18.806869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:18.818992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd45b2e4-7424-4842-8506-f673d94ce291 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:18.854310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a07912-8f06-4c7c-ace6-9148c95a3afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:18.855072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a07912-8f06-4c7c-ace6-9148c95a3afe None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.855381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a07912-8f06-4c7c-ace6-9148c95a3afe None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.855666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a07912-8f06-4c7c-ace6-9148c95a3afe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.866464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a07912-8f06-4c7c-ace6-9148c95a3afe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-00a07912-8f06-4c7c-ace6-9148c95a3afe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:18.874590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:18.875552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:18.876343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 97/190] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:18.876993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a07912-8f06-4c7c-ace6-9148c95a3afe None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:18.898065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 94/191] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 20 08:59:18.899420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688525-6171-4c8d-b723-282c6926cfee None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:18.899702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688525-6171-4c8d-b723-282c6926cfee None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:18.899970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688525-6171-4c8d-b723-282c6926cfee None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:18.901373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 95/192] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:18.903556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17688525-6171-4c8d-b723-282c6926cfee None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:18.973270 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a07912-8f06-4c7c-ace6-9148c95a3afe None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:18.999546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17688525-6171-4c8d-b723-282c6926cfee None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.006224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090e336-93e3-4a0d-9160-cb4f986a1745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.007011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090e336-93e3-4a0d-9160-cb4f986a1745 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.007297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090e336-93e3-4a0d-9160-cb4f986a1745 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.007613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4090e336-93e3-4a0d-9160-cb4f986a1745 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.008796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090e336-93e3-4a0d-9160-cb4f986a1745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.010253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4090e336-93e3-4a0d-9160-cb4f986a1745 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.012044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 98/193] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:19.012634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090e336-93e3-4a0d-9160-cb4f986a1745 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.015934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:19.021152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:19.038457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e95a2-2f8e-46aa-b648-41c99c1b699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.039121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e95a2-2f8e-46aa-b648-41c99c1b699b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.039439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e95a2-2f8e-46aa-b648-41c99c1b699b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.039713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e95a2-2f8e-46aa-b648-41c99c1b699b None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.040469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e95a2-2f8e-46aa-b648-41c99c1b699b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:19.041087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e95a2-2f8e-46aa-b648-41c99c1b699b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.042484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e95a2-2f8e-46aa-b648-41c99c1b699b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.050699 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 96/194] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:19.051923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af6e2-2d79-48d0-b93a-9cbb394f8eae None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.052211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af6e2-2d79-48d0-b93a-9cbb394f8eae None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.052486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2af6e2-2d79-48d0-b93a-9cbb394f8eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.068274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-dc2af6e2-2d79-48d0-b93a-9cbb394f8eae None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:19.069443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-dc2af6e2-2d79-48d0-b93a-9cbb394f8eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:19.113794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4090e336-93e3-4a0d-9160-cb4f986a1745 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.135245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 97/195] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:19.136078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2af6e2-2d79-48d0-b93a-9cbb394f8eae None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.157536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b42414-c727-4691-864f-2cca087746e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.158409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b42414-c727-4691-864f-2cca087746e1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.158695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b42414-c727-4691-864f-2cca087746e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.159017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b42414-c727-4691-864f-2cca087746e1 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.160298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b42414-c727-4691-864f-2cca087746e1 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.161749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b42414-c727-4691-864f-2cca087746e1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.163444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 99/196] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:19.164197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b42414-c727-4691-864f-2cca087746e1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.231485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:19.238831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2af6e2-2d79-48d0-b93a-9cbb394f8eae None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.249411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:19.261252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b42414-c727-4691-864f-2cca087746e1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.273102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47911b68-f9fc-408c-99da-184c9d7a335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.273806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47911b68-f9fc-408c-99da-184c9d7a335a None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.274092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47911b68-f9fc-408c-99da-184c9d7a335a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.274369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47911b68-f9fc-408c-99da-184c9d7a335a None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.275086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47911b68-f9fc-408c-99da-184c9d7a335a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.276567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47911b68-f9fc-408c-99da-184c9d7a335a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.296613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eca867-7ad2-40d7-851f-3341b6d4ef93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.297181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 98/197] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:19 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:19.297568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eca867-7ad2-40d7-851f-3341b6d4ef93 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.297858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eca867-7ad2-40d7-851f-3341b6d4ef93 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.298029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47911b68-f9fc-408c-99da-184c9d7a335a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.298196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eca867-7ad2-40d7-851f-3341b6d4ef93 None admin] PATH_INFO: `/v3/users/cf16d55421fb44f99dedfb8d0d28ced6` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.299459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eca867-7ad2-40d7-851f-3341b6d4ef93 None admin] RBAC: Authorizing `identity:get_user(user_id=cf16d55421fb44f99dedfb8d0d28ced6)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.300943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eca867-7ad2-40d7-851f-3341b6d4ef93 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.302669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 100/198] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/users/cf16d55421fb44f99dedfb8d0d28ced6 => generated 283 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:19.303497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eca867-7ad2-40d7-851f-3341b6d4ef93 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.419087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eca867-7ad2-40d7-851f-3341b6d4ef93 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.421693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47911b68-f9fc-408c-99da-184c9d7a335a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.456748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.457425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.457647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc737a-9ce4-4f40-94b6-bf7ffc0eb370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.457760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.457969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.458414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc737a-9ce4-4f40-94b6-bf7ffc0eb370 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.458692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc737a-9ce4-4f40-94b6-bf7ffc0eb370 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.458874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=efcdcc966ddd493f83f40b196e076f1a, user.id=cf16d55421fb44f99dedfb8d0d28ced6) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:19.458969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebc737a-9ce4-4f40-94b6-bf7ffc0eb370 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.459127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.460630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.467145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 101/199] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/role_assignments?user.id=cf16d55421fb44f99dedfb8d0d28ced6&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=6dca675ade814980a252f800fb4fa3ad => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:19.467903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.470838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebc737a-9ce4-4f40-94b6-bf7ffc0eb370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:19.472439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-4ebc737a-9ce4-4f40-94b6-bf7ffc0eb370 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:19.478839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:19.479904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:19.480694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 99/200] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:19.487691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc737a-9ce4-4f40-94b6-bf7ffc0eb370 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.521269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:19.617612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a308dfc-20d9-4494-b30a-85927c0b483a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.632508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebc737a-9ce4-4f40-94b6-bf7ffc0eb370 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.663974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43fb0-6578-4ad1-a3e5-61b905b1090e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.664696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43fb0-6578-4ad1-a3e5-61b905b1090e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.664984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43fb0-6578-4ad1-a3e5-61b905b1090e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.665306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d43fb0-6578-4ad1-a3e5-61b905b1090e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.677476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.678080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.678362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.679112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.679941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:19.680257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.682076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.698400 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 100/201] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:19.705506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.748655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d43fb0-6578-4ad1-a3e5-61b905b1090e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.751135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d43fb0-6578-4ad1-a3e5-61b905b1090e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.763694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:19.827740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:19.830632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad39b43a-502e-4a6a-b7e7-76f30e5b2f15 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:19.871062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 102/202] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:19.878449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43fb0-6578-4ad1-a3e5-61b905b1090e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:19.884905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf451ed-96d6-4a91-b786-5cab821f63d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:19.885791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf451ed-96d6-4a91-b786-5cab821f63d1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:19.886069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf451ed-96d6-4a91-b786-5cab821f63d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:19.886341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf451ed-96d6-4a91-b786-5cab821f63d1 None admin] PATH_INFO: `/v3/users/468703f0020b4014bb0b77a4767056b1` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:19.889683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf451ed-96d6-4a91-b786-5cab821f63d1 None admin] RBAC: Authorizing `identity:get_user(user_id=468703f0020b4014bb0b77a4767056b1)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:19.891164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf451ed-96d6-4a91-b786-5cab821f63d1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:19.893096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 101/203] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/users/468703f0020b4014bb0b77a4767056b1 => generated 289 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:19.903319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf451ed-96d6-4a91-b786-5cab821f63d1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:20.017197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf451ed-96d6-4a91-b786-5cab821f63d1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:20.042027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d43fb0-6578-4ad1-a3e5-61b905b1090e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:20.059398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:20.063954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a34ba9-db5b-4308-a992-d5059d3d173c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:20.064709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a34ba9-db5b-4308-a992-d5059d3d173c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:20.065055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a34ba9-db5b-4308-a992-d5059d3d173c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:20.065397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a34ba9-db5b-4308-a992-d5059d3d173c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:20.084914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946248bf-617b-4cce-a93c-572ede495a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:20.085144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946248bf-617b-4cce-a93c-572ede495a28 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:20.085439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946248bf-617b-4cce-a93c-572ede495a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:20.087440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946248bf-617b-4cce-a93c-572ede495a28 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:20.089165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946248bf-617b-4cce-a93c-572ede495a28 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:20.090804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946248bf-617b-4cce-a93c-572ede495a28 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:20.097156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 103/204] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:20.100403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946248bf-617b-4cce-a93c-572ede495a28 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:20.108910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:20.176421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a34ba9-db5b-4308-a992-d5059d3d173c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:20.178300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a34ba9-db5b-4308-a992-d5059d3d173c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:20.260758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946248bf-617b-4cce-a93c-572ede495a28 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:20.303226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:20.304258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:20.304789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:20.305284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:20.306366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] RBAC: Adding query filter params (role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=efcdcc966ddd493f83f40b196e076f1a, user.id=cf16d55421fb44f99dedfb8d0d28ced6) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:20.306892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:20.308720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:20.315022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 104/205] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:20 2022] GET /identity/v3/role_assignments?user.id=cf16d55421fb44f99dedfb8d0d28ced6&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:20.379713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 102/206] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:20.385489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:20.445542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:20.534596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53913b5-568a-4ab4-b2c8-eacd2d956980 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:20.579619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826a84e-2e65-454e-bda4-73c03fe689da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:20.580415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a84e-2e65-454e-bda4-73c03fe689da None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:20.580696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a84e-2e65-454e-bda4-73c03fe689da None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:20.580970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826a84e-2e65-454e-bda4-73c03fe689da None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:20.582157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826a84e-2e65-454e-bda4-73c03fe689da None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:20.583646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826a84e-2e65-454e-bda4-73c03fe689da None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:20.585681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 105/207] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:20 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:20.591410 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a34ba9-db5b-4308-a992-d5059d3d173c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:20.665653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:20.730917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a34ba9-db5b-4308-a992-d5059d3d173c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:20.784029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd95d807-2946-47af-b6be-f4bfecb4ff0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:20.784957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d807-2946-47af-b6be-f4bfecb4ff0d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:20.785439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d807-2946-47af-b6be-f4bfecb4ff0d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:20.785887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd95d807-2946-47af-b6be-f4bfecb4ff0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:20.786907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95d807-2946-47af-b6be-f4bfecb4ff0d None admin] RBAC: Adding query filter params (role.id=dd10b97be2224165a8d312ca6eefdbfd, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7, user.id=468703f0020b4014bb0b77a4767056b1) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:20.787416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95d807-2946-47af-b6be-f4bfecb4ff0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:20.789034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd95d807-2946-47af-b6be-f4bfecb4ff0d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:20.800485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 103/208] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:20 2022] GET /identity/v3/role_assignments?user.id=468703f0020b4014bb0b77a4767056b1&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=dd10b97be2224165a8d312ca6eefdbfd => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:21.930004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3f36-ae67-4e38-8f52-f15f17b7e100 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:21.930516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3f36-ae67-4e38-8f52-f15f17b7e100 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:21.930590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e3f36-ae67-4e38-8f52-f15f17b7e100 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:21.932252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 106/209] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:21.937597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59487e9-6d41-40b4-a809-ee72226559b9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:21.937975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59487e9-6d41-40b4-a809-ee72226559b9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:21.938155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59487e9-6d41-40b4-a809-ee72226559b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:21.955026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f59487e9-6d41-40b4-a809-ee72226559b9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:21.956339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f59487e9-6d41-40b4-a809-ee72226559b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:21.966022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c758b-e9da-40e6-a0ef-8bd1bd5aac71 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:21.966317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c758b-e9da-40e6-a0ef-8bd1bd5aac71 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:21.966599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6c758b-e9da-40e6-a0ef-8bd1bd5aac71 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:21.968028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 107/210] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:21.973071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b0274-b729-4c18-963b-185cdcfcef68 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:21.973349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b0274-b729-4c18-963b-185cdcfcef68 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:21.973625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b0274-b729-4c18-963b-185cdcfcef68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:21.990729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-fb5b0274-b729-4c18-963b-185cdcfcef68 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:21.992032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fb5b0274-b729-4c18-963b-185cdcfcef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:22.033264 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 104/211] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:22.056957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59487e9-6d41-40b4-a809-ee72226559b9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.063896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 108/212] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:22.087650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b0274-b729-4c18-963b-185cdcfcef68 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.111872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:22.124094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:22.173952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59487e9-6d41-40b4-a809-ee72226559b9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.197131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b0274-b729-4c18-963b-185cdcfcef68 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.210222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669ace9-f312-4982-aebd-02ba9cb914d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.211010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669ace9-f312-4982-aebd-02ba9cb914d5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.211369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669ace9-f312-4982-aebd-02ba9cb914d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.211648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669ace9-f312-4982-aebd-02ba9cb914d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.222581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669ace9-f312-4982-aebd-02ba9cb914d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-f669ace9-f312-4982-aebd-02ba9cb914d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:22.230339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:22.231208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:22.232062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 105/213] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:22.233376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0608d71-f944-4918-85eb-5cb59e7a645c None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.233376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0608d71-f944-4918-85eb-5cb59e7a645c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.233376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0608d71-f944-4918-85eb-5cb59e7a645c None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.234705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 106/214] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:22.239068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78427646-3ace-4ebf-9486-6530d76ef9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.239212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0608d71-f944-4918-85eb-5cb59e7a645c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.240020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78427646-3ace-4ebf-9486-6530d76ef9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.240300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78427646-3ace-4ebf-9486-6530d76ef9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.240568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78427646-3ace-4ebf-9486-6530d76ef9c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.241947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78427646-3ace-4ebf-9486-6530d76ef9c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:22.244251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78427646-3ace-4ebf-9486-6530d76ef9c2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:22.246062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 109/215] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:22.247142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ea26-9907-4bd6-9b6a-77d813a9a8db None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.247452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ea26-9907-4bd6-9b6a-77d813a9a8db None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.247731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc3ea26-9907-4bd6-9b6a-77d813a9a8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.266704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-cdc3ea26-9907-4bd6-9b6a-77d813a9a8db None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:22.267895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-cdc3ea26-9907-4bd6-9b6a-77d813a9a8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:22.338252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 110/216] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:22.339347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3ea26-9907-4bd6-9b6a-77d813a9a8db None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.360786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0608d71-f944-4918-85eb-5cb59e7a645c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.373202 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:22.374301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:22.398768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.399466 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.399745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.400015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.400766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:22.401067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:22.402485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:22.410800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 107/217] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:22.411526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.450102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc3ea26-9907-4bd6-9b6a-77d813a9a8db None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.484087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199bda4-9de6-479d-b36c-663774756fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.484767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199bda4-9de6-479d-b36c-663774756fed None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.485098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199bda4-9de6-479d-b36c-663774756fed None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.485379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199bda4-9de6-479d-b36c-663774756fed None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.486101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199bda4-9de6-479d-b36c-663774756fed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:22.487682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a199bda4-9de6-479d-b36c-663774756fed None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:22.519301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 111/218] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:22 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:22.519301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199bda4-9de6-479d-b36c-663774756fed None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.535116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b01ad8-42b1-4f5b-8061-e29261d87d8b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.573051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370ba11-9a31-4f64-942c-140638a0036b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.574083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370ba11-9a31-4f64-942c-140638a0036b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.574454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370ba11-9a31-4f64-942c-140638a0036b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.574825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9370ba11-9a31-4f64-942c-140638a0036b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.587285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9370ba11-9a31-4f64-942c-140638a0036b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-9370ba11-9a31-4f64-942c-140638a0036b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:22.596313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:22.597297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:22.598084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 108/219] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:22.604988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370ba11-9a31-4f64-942c-140638a0036b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.626722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:22.642108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199bda4-9de6-479d-b36c-663774756fed None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.644932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:22.683706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ab6f5-af9f-42cf-bcb2-fbef2ebcb465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.684743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ab6f5-af9f-42cf-bcb2-fbef2ebcb465 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.685028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ab6f5-af9f-42cf-bcb2-fbef2ebcb465 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.685305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ab6f5-af9f-42cf-bcb2-fbef2ebcb465 None admin] PATH_INFO: `/v3/users/cf16d55421fb44f99dedfb8d0d28ced6` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.686812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ab6f5-af9f-42cf-bcb2-fbef2ebcb465 None admin] RBAC: Authorizing `identity:get_user(user_id=cf16d55421fb44f99dedfb8d0d28ced6)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:22.688504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ab6f5-af9f-42cf-bcb2-fbef2ebcb465 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:22.695334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 112/220] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:22 2022] GET /identity/v3/users/cf16d55421fb44f99dedfb8d0d28ced6 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:22.696520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ab6f5-af9f-42cf-bcb2-fbef2ebcb465 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.737112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9370ba11-9a31-4f64-942c-140638a0036b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.775018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.775791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.776076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.776351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.777174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] RBAC: Adding query filter params (name=member) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:22.777487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:22.778899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:22.794955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 109/221] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:22.801427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.809584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ab6f5-af9f-42cf-bcb2-fbef2ebcb465 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.847978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a5779-33f3-4b7c-9808-4ffc3a78131f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.848773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5779-33f3-4b7c-9808-4ffc3a78131f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.849048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5779-33f3-4b7c-9808-4ffc3a78131f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.849314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71a5779-33f3-4b7c-9808-4ffc3a78131f None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.850541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a5779-33f3-4b7c-9808-4ffc3a78131f None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:22.852627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71a5779-33f3-4b7c-9808-4ffc3a78131f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:22.855549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 113/222] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:22 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:22.860817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a5779-33f3-4b7c-9808-4ffc3a78131f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:22.896275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:22.933780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:22.941773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199aaae3-6510-42de-9b4d-18da5a78c54f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:22.980329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a85f88-ff82-425a-a84d-ed123feaa34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:22.981306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a85f88-ff82-425a-a84d-ed123feaa34c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:22.981938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a85f88-ff82-425a-a84d-ed123feaa34c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:22.981938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a85f88-ff82-425a-a84d-ed123feaa34c None admin] PATH_INFO: `/v3/users/b28eecf89c8a402a963a4e1bcf0b294b` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:22.996676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a85f88-ff82-425a-a84d-ed123feaa34c None admin] RBAC: Authorizing `identity:get_user(user_id=b28eecf89c8a402a963a4e1bcf0b294b)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:22.998243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a85f88-ff82-425a-a84d-ed123feaa34c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:22.999168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71a5779-33f3-4b7c-9808-4ffc3a78131f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:23.004689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 110/223] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:22 2022] GET /identity/v3/users/b28eecf89c8a402a963a4e1bcf0b294b => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:23.006144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba85c9-e5d4-41a5-8f32-c98d2bac6174 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.006494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba85c9-e5d4-41a5-8f32-c98d2bac6174 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.006836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ba85c9-e5d4-41a5-8f32-c98d2bac6174 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.008551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 111/224] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:23.009404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba85c9-e5d4-41a5-8f32-c98d2bac6174 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:23.037656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b903f3-d099-4120-809b-5838e0693001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:23.039009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b903f3-d099-4120-809b-5838e0693001 None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.039519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b903f3-d099-4120-809b-5838e0693001 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.039970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b903f3-d099-4120-809b-5838e0693001 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a/users/cf16d55421fb44f99dedfb8d0d28ced6/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.042601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b903f3-d099-4120-809b-5838e0693001 None admin] RBAC: Authorizing `identity:create_grant(project_id=efcdcc966ddd493f83f40b196e076f1a, user_id=cf16d55421fb44f99dedfb8d0d28ced6, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:23.044392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b903f3-d099-4120-809b-5838e0693001 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:23.063423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 114/225] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:22 2022] PUT /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a/users/cf16d55421fb44f99dedfb8d0d28ced6/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:23.065337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a797e97-4c3a-4862-b273-706ef92f1470 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.065875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a797e97-4c3a-4862-b273-706ef92f1470 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.066383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a797e97-4c3a-4862-b273-706ef92f1470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.085922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4a797e97-4c3a-4862-b273-706ef92f1470 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:23.087823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4a797e97-4c3a-4862-b273-706ef92f1470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:23.122498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ba85c9-e5d4-41a5-8f32-c98d2bac6174 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:23.163859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464bdf-1274-4cdb-9617-d0a26cbf7b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:23.164956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464bdf-1274-4cdb-9617-d0a26cbf7b6d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.165189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464bdf-1274-4cdb-9617-d0a26cbf7b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.165374 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88464bdf-1274-4cdb-9617-d0a26cbf7b6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.170452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 115/226] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:23.190272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:23.211633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a797e97-4c3a-4862-b273-706ef92f1470 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:23.237515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:23.240957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88464bdf-1274-4cdb-9617-d0a26cbf7b6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:23.247294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88464bdf-1274-4cdb-9617-d0a26cbf7b6d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:23.391233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 112/227] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 20 08:59:23.394289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a797e97-4c3a-4862-b273-706ef92f1470 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:23.397633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464bdf-1274-4cdb-9617-d0a26cbf7b6d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:23.434600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7625a-e282-4e0d-8ca7-6d02fe23ec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:23.435497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7625a-e282-4e0d-8ca7-6d02fe23ec7c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.435785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7625a-e282-4e0d-8ca7-6d02fe23ec7c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.436280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7625a-e282-4e0d-8ca7-6d02fe23ec7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.453952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7625a-e282-4e0d-8ca7-6d02fe23ec7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-25b7625a-e282-4e0d-8ca7-6d02fe23ec7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:23.462445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:23.463367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:23.464184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 116/228] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:23.487612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:23.487793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7625a-e282-4e0d-8ca7-6d02fe23ec7c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:23.533595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88464bdf-1274-4cdb-9617-d0a26cbf7b6d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:23.599717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:23.622688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7625a-e282-4e0d-8ca7-6d02fe23ec7c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:23.636796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83095702-b2e9-4226-b62b-02d40c534b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:23.637737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83095702-b2e9-4226-b62b-02d40c534b5d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.638018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83095702-b2e9-4226-b62b-02d40c534b5d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.638292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83095702-b2e9-4226-b62b-02d40c534b5d None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.639736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83095702-b2e9-4226-b62b-02d40c534b5d None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:23.641219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83095702-b2e9-4226-b62b-02d40c534b5d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:23.643017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 113/229] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:23 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:23.652316 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83095702-b2e9-4226-b62b-02d40c534b5d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:23.662184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:23.662869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.663153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.663449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.664244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:23.664555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:23.666330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:23.674982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 117/230] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:23.681223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:23.792867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15cf81f-9151-4aff-a038-fcf67e235f98 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:23.835811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6f58d-f5df-4055-ba30-6ac737221215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:23.836781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f58d-f5df-4055-ba30-6ac737221215 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.837042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f58d-f5df-4055-ba30-6ac737221215 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.837313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6f58d-f5df-4055-ba30-6ac737221215 None admin] PATH_INFO: `/v3/users/cf16d55421fb44f99dedfb8d0d28ced6` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.839889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6f58d-f5df-4055-ba30-6ac737221215 None admin] RBAC: Authorizing `identity:get_user(user_id=cf16d55421fb44f99dedfb8d0d28ced6)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:23.841349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6f58d-f5df-4055-ba30-6ac737221215 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:23.856547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 118/231] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:23 2022] GET /identity/v3/users/cf16d55421fb44f99dedfb8d0d28ced6 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:23.857443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6f58d-f5df-4055-ba30-6ac737221215 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:23.871160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:23.892078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83095702-b2e9-4226-b62b-02d40c534b5d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:23.913672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:23.936704 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:23.937365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:23.937641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:23.937914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:23.938749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] RBAC: Adding query filter params (role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=efcdcc966ddd493f83f40b196e076f1a, user.id=b28eecf89c8a402a963a4e1bcf0b294b) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:23.939058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:23.940516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:23.954608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 114/232] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:23 2022] GET /identity/v3/role_assignments?user.id=b28eecf89c8a402a963a4e1bcf0b294b&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 259 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:24.007857 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6f58d-f5df-4055-ba30-6ac737221215 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:24.051804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0db6f7-48cf-465a-a6c1-28eb806d48ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:24.052700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0db6f7-48cf-465a-a6c1-28eb806d48ad None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:24.052986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0db6f7-48cf-465a-a6c1-28eb806d48ad None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:24.053265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0db6f7-48cf-465a-a6c1-28eb806d48ad None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:24.056071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0db6f7-48cf-465a-a6c1-28eb806d48ad None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:24.057562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0db6f7-48cf-465a-a6c1-28eb806d48ad None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:24.059615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 119/233] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:23 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:24.065286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:24.193812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36967f11-5fcd-4db5-9163-b25bf1a2c720 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:24.229140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd41ce51-5b42-4a99-b0d9-445b0cbd8bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:24.230035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41ce51-5b42-4a99-b0d9-445b0cbd8bf8 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:24.230312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41ce51-5b42-4a99-b0d9-445b0cbd8bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:24.230582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd41ce51-5b42-4a99-b0d9-445b0cbd8bf8 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a/users/cf16d55421fb44f99dedfb8d0d28ced6/roles/2e550a6a96bb4804926c6213a5af7585` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:24.232842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41ce51-5b42-4a99-b0d9-445b0cbd8bf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=efcdcc966ddd493f83f40b196e076f1a, user_id=cf16d55421fb44f99dedfb8d0d28ced6, role_id=2e550a6a96bb4804926c6213a5af7585)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:24.234301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd41ce51-5b42-4a99-b0d9-445b0cbd8bf8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:24.255329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 115/234] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:24 2022] PUT /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a/users/cf16d55421fb44f99dedfb8d0d28ced6/roles/2e550a6a96bb4804926c6213a5af7585 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:25.353970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f2fdf-527f-4b9d-943f-af54c9398602 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.354518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f2fdf-527f-4b9d-943f-af54c9398602 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.354518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f2fdf-527f-4b9d-943f-af54c9398602 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.355931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 120/235] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:25.361189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989573c-289b-4ad3-b08e-4f2638bd85f4 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.361574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989573c-289b-4ad3-b08e-4f2638bd85f4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.361751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989573c-289b-4ad3-b08e-4f2638bd85f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.378859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b989573c-289b-4ad3-b08e-4f2638bd85f4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:25.380077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b989573c-289b-4ad3-b08e-4f2638bd85f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:25.412191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8ca9c-8413-4bd6-89f9-d2f2be55140b None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.412476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8ca9c-8413-4bd6-89f9-d2f2be55140b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.412760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da8ca9c-8413-4bd6-89f9-d2f2be55140b None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.414479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 121/236] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:25.419520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731b540-5c49-4e79-aec0-62fd31115eee None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.419809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731b540-5c49-4e79-aec0-62fd31115eee None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.420081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731b540-5c49-4e79-aec0-62fd31115eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.447674 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:25.454326 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9731b540-5c49-4e79-aec0-62fd31115eee None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:25.456619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9731b540-5c49-4e79-aec0-62fd31115eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:25.468630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 116/237] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:25.510010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989573c-289b-4ad3-b08e-4f2638bd85f4 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:25.563547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 122/238] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:25.628049 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731b540-5c49-4e79-aec0-62fd31115eee None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:25.630288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:25.685176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989573c-289b-4ad3-b08e-4f2638bd85f4 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:25.754361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ac38f-052f-4cf0-866a-810a8decf035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:25.755220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ac38f-052f-4cf0-866a-810a8decf035 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.755536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ac38f-052f-4cf0-866a-810a8decf035 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.755805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ac38f-052f-4cf0-866a-810a8decf035 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.789707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ac38f-052f-4cf0-866a-810a8decf035 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:25.797168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731b540-5c49-4e79-aec0-62fd31115eee None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-5c7ac38f-052f-4cf0-866a-810a8decf035 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:25.799328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:25.800183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:25.800970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:25.802008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 117/239] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:25.802906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ce338-e053-438d-8e6a-e085501702af None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.803126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ce338-e053-438d-8e6a-e085501702af None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.803422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ce338-e053-438d-8e6a-e085501702af None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.804668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 118/240] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:25.810524 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ce338-e053-438d-8e6a-e085501702af None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:25.845628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95523f-f014-416d-a0e5-ad4505765931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:25.846526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95523f-f014-416d-a0e5-ad4505765931 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.846811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95523f-f014-416d-a0e5-ad4505765931 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.847086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95523f-f014-416d-a0e5-ad4505765931 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.848389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95523f-f014-416d-a0e5-ad4505765931 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:25.849965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95523f-f014-416d-a0e5-ad4505765931 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:25.851771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 123/241] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:25.857413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef74208-a526-4725-8bff-68f1c5a287bf None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.857776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:25.858288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef74208-a526-4725-8bff-68f1c5a287bf None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.858577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef74208-a526-4725-8bff-68f1c5a287bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.879470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9ef74208-a526-4725-8bff-68f1c5a287bf None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:25.881039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9ef74208-a526-4725-8bff-68f1c5a287bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:25.949194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ce338-e053-438d-8e6a-e085501702af None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:25.956724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:25.962593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 124/242] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:25.964740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef74208-a526-4725-8bff-68f1c5a287bf None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:25.990900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:25.991635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:25.992148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:25.992490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:25.993262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:25.993572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:25.995503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:26.010734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 119/243] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:26.010734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:26.078277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef74208-a526-4725-8bff-68f1c5a287bf None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:26.114721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4a084-b637-4535-9c42-4eb0a3030a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:26.115815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a084-b637-4535-9c42-4eb0a3030a2e None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.116338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a084-b637-4535-9c42-4eb0a3030a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.116805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d4a084-b637-4535-9c42-4eb0a3030a2e None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.117858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4a084-b637-4535-9c42-4eb0a3030a2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:26.119603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d4a084-b637-4535-9c42-4eb0a3030a2e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:26.121150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:26.128356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4071a45-2cda-4739-9e0c-9d850f825717 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:26.146915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 125/244] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:25 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:26.156354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4a084-b637-4535-9c42-4eb0a3030a2e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:26.169293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf2f51-cd0e-48a6-a398-96aacfbe810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:26.170503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf2f51-cd0e-48a6-a398-96aacfbe810e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.170980 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf2f51-cd0e-48a6-a398-96aacfbe810e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.171513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf2f51-cd0e-48a6-a398-96aacfbe810e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.185007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf2f51-cd0e-48a6-a398-96aacfbe810e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-21bf2f51-cd0e-48a6-a398-96aacfbe810e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:26.193705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:26.194956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:26.196084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:26.198642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 120/245] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:26.207310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf2f51-cd0e-48a6-a398-96aacfbe810e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:26.304778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:26.334506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf2f51-cd0e-48a6-a398-96aacfbe810e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:26.374853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:26.377315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.377764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.378176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.379142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:26.379607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:26.381163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:26.391410 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 121/246] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:26.397808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:26.454707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:26.462971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d4a084-b637-4535-9c42-4eb0a3030a2e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:26.507846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f6b18-1eb9-47ee-bb4b-ce069a9a296a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:26.508811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f6b18-1eb9-47ee-bb4b-ce069a9a296a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.509147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f6b18-1eb9-47ee-bb4b-ce069a9a296a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.509426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f6b18-1eb9-47ee-bb4b-ce069a9a296a None admin] PATH_INFO: `/v3/users/c340785b17254d15887d496fef472f90` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.525489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f6b18-1eb9-47ee-bb4b-ce069a9a296a None admin] RBAC: Authorizing `identity:get_user(user_id=c340785b17254d15887d496fef472f90)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:26.526980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f6b18-1eb9-47ee-bb4b-ce069a9a296a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:26.528808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724e18-f47e-4894-bdf1-0d1fc9f63381 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:26.528927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 126/247] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:26 2022] GET /identity/v3/users/c340785b17254d15887d496fef472f90 => generated 297 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:26.534314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f6b18-1eb9-47ee-bb4b-ce069a9a296a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:26.569911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa74e2-a663-4940-8435-7a90fe762e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:26.572285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa74e2-a663-4940-8435-7a90fe762e38 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.572712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa74e2-a663-4940-8435-7a90fe762e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.573122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa74e2-a663-4940-8435-7a90fe762e38 None admin] PATH_INFO: `/v3/users/cf16d55421fb44f99dedfb8d0d28ced6` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.574546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa74e2-a663-4940-8435-7a90fe762e38 None admin] RBAC: Authorizing `identity:get_user(user_id=cf16d55421fb44f99dedfb8d0d28ced6)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:26.576180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa74e2-a663-4940-8435-7a90fe762e38 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:26.578925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 122/248] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:26 2022] GET /identity/v3/users/cf16d55421fb44f99dedfb8d0d28ced6 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:26.584665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa74e2-a663-4940-8435-7a90fe762e38 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:26.663420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f6b18-1eb9-47ee-bb4b-ce069a9a296a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:26.680806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:26.701644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:26.711974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0086af-7344-44f0-8612-c517009ade04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:26.712856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0086af-7344-44f0-8612-c517009ade04 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.713277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0086af-7344-44f0-8612-c517009ade04 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.713667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa74e2-a663-4940-8435-7a90fe762e38 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:26.713851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0086af-7344-44f0-8612-c517009ade04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.752114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba28854-3b4b-4fa3-85ac-e5a1aa99a53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:26.752437 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba28854-3b4b-4fa3-85ac-e5a1aa99a53f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.752716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba28854-3b4b-4fa3-85ac-e5a1aa99a53f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.752986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba28854-3b4b-4fa3-85ac-e5a1aa99a53f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.813208 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0086af-7344-44f0-8612-c517009ade04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:26.815038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0086af-7344-44f0-8612-c517009ade04 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:26.853315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba28854-3b4b-4fa3-85ac-e5a1aa99a53f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:26.854804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba28854-3b4b-4fa3-85ac-e5a1aa99a53f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:26.962849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:26.972072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 127/249] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 20 08:59:26.973930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea90d-5f1e-4c7a-bf3d-4869eee23202 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.974257 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea90d-5f1e-4c7a-bf3d-4869eee23202 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.978918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eea90d-5f1e-4c7a-bf3d-4869eee23202 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.980342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 128/250] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:26.981581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4993bc-f95e-4761-9d7a-fd9efb65cf8f None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.981839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4993bc-f95e-4761-9d7a-fd9efb65cf8f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.982121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4993bc-f95e-4761-9d7a-fd9efb65cf8f None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:26.983608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 129/251] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:26.984345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4993bc-f95e-4761-9d7a-fd9efb65cf8f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:26.993444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 123/252] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:26.995948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a91e-7cfb-4b9c-85e3-fd1912a633e5 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:26.996240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a91e-7cfb-4b9c-85e3-fd1912a633e5 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:26.996517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d2a91e-7cfb-4b9c-85e3-fd1912a633e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.007003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:27.017706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-30d2a91e-7cfb-4b9c-85e3-fd1912a633e5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:27.018905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-30d2a91e-7cfb-4b9c-85e3-fd1912a633e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:27.089404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 124/253] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:27.090826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393f2d7-c84a-4b90-9fde-54026e3bf991 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:27.091114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393f2d7-c84a-4b90-9fde-54026e3bf991 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:27.091424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393f2d7-c84a-4b90-9fde-54026e3bf991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.096072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4993bc-f95e-4761-9d7a-fd9efb65cf8f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:27.109700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3393f2d7-c84a-4b90-9fde-54026e3bf991 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:27.110895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3393f2d7-c84a-4b90-9fde-54026e3bf991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:27.132107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d981-d9b1-440d-b4d6-8c34d261bd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:27.133105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d981-d9b1-440d-b4d6-8c34d261bd39 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:27.133388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d981-d9b1-440d-b4d6-8c34d261bd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:27.133664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d981-d9b1-440d-b4d6-8c34d261bd39 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.135204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943d981-d9b1-440d-b4d6-8c34d261bd39 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:27.136708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943d981-d9b1-440d-b4d6-8c34d261bd39 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:27.143161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 130/254] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:26 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:27.143161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d981-d9b1-440d-b4d6-8c34d261bd39 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:27.207534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 125/255] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:27.208470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393f2d7-c84a-4b90-9fde-54026e3bf991 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:27.242744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:27.267973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d981-d9b1-440d-b4d6-8c34d261bd39 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:27.273678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:27.313358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e65598-e92d-4d71-a635-c6d21a4d1eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:27.314163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65598-e92d-4d71-a635-c6d21a4d1eca None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:27.314445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65598-e92d-4d71-a635-c6d21a4d1eca None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:27.314720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e65598-e92d-4d71-a635-c6d21a4d1eca None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.315941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e65598-e92d-4d71-a635-c6d21a4d1eca None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:27.317420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e65598-e92d-4d71-a635-c6d21a4d1eca None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:27.319330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 131/256] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:27 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:27.320041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e65598-e92d-4d71-a635-c6d21a4d1eca None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:27.339552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393f2d7-c84a-4b90-9fde-54026e3bf991 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:27.375899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b6b98-2fe3-415f-9a03-bb0108ed2ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:27.376749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b6b98-2fe3-415f-9a03-bb0108ed2ccb None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:27.377072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b6b98-2fe3-415f-9a03-bb0108ed2ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:27.377357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b6b98-2fe3-415f-9a03-bb0108ed2ccb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.388067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b6b98-2fe3-415f-9a03-bb0108ed2ccb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8e0b6b98-2fe3-415f-9a03-bb0108ed2ccb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:27.395986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:27.396877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:27.397683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 126/257] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:27.398861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b6b98-2fe3-415f-9a03-bb0108ed2ccb None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:27.448062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e65598-e92d-4d71-a635-c6d21a4d1eca None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:27.486014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:27.487307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:27.487600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:27.487978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.489319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] RBAC: Adding query filter params (role.id=dd10b97be2224165a8d312ca6eefdbfd, scope.project.id=efcdcc966ddd493f83f40b196e076f1a, user.id=c340785b17254d15887d496fef472f90) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:27.489937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:27.491614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:27.503513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 132/258] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:27 2022] GET /identity/v3/role_assignments?user.id=c340785b17254d15887d496fef472f90&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=dd10b97be2224165a8d312ca6eefdbfd => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:27.503513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:27.533510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:27.536353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b6b98-2fe3-415f-9a03-bb0108ed2ccb None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:27.561024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:27.638911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c48c3e-08ba-4488-9bd7-ff276db8ce42 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:27.668139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327cda3-310c-4f2a-9c25-f79087e0f6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:27.668933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327cda3-310c-4f2a-9c25-f79087e0f6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:27.669212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327cda3-310c-4f2a-9c25-f79087e0f6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:27.669484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b327cda3-310c-4f2a-9c25-f79087e0f6b8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.688022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:27.689074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:27.689557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:27.690054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.691132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=efcdcc966ddd493f83f40b196e076f1a, user.id=cf16d55421fb44f99dedfb8d0d28ced6) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:27.691676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:27.693364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:27.697729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b327cda3-310c-4f2a-9c25-f79087e0f6b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:27.707636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 133/259] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:27 2022] GET /identity/v3/role_assignments?user.id=cf16d55421fb44f99dedfb8d0d28ced6&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=6dca675ade814980a252f800fb4fa3ad => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:27.712313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b327cda3-310c-4f2a-9c25-f79087e0f6b8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:27.712938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:27.713842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:27.714706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:27.717511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 127/260] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:27.725789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327cda3-310c-4f2a-9c25-f79087e0f6b8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:27.907362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b327cda3-310c-4f2a-9c25-f79087e0f6b8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:27.947439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:27.989892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:27.990582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:27.990860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:27.991132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:27.991923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:27.992223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:27.993687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:28.009495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331188b2-bf60-4302-aaa9-ced2937111bf None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:28.011528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 128/261] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:28.033539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:28.088926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:28.092873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:28.093625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:28.093914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:28.094179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:28.094945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:28.095296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:28.098113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:28.115815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 134/262] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:28.121436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:28.218813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6976d9e-ecd1-41b3-890e-d1e3586cbd05 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:28.277827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436c0d-2d95-4606-b877-176f38a0f687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:28.278701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436c0d-2d95-4606-b877-176f38a0f687 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:28.278973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436c0d-2d95-4606-b877-176f38a0f687 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:28.279238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd436c0d-2d95-4606-b877-176f38a0f687 None admin] PATH_INFO: `/v3/users/cf16d55421fb44f99dedfb8d0d28ced6` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:28.281055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd436c0d-2d95-4606-b877-176f38a0f687 None admin] RBAC: Authorizing `identity:get_user(user_id=cf16d55421fb44f99dedfb8d0d28ced6)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:28.282516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd436c0d-2d95-4606-b877-176f38a0f687 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:28.284358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 129/263] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:28 2022] GET /identity/v3/users/cf16d55421fb44f99dedfb8d0d28ced6 => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:28.289244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf0bc4d-d956-4645-b2a8-ec3d44573984 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:28.291786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436c0d-2d95-4606-b877-176f38a0f687 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:28.384062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43253b79-69a3-47ce-89a8-7be9aeda0eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:28.385328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43253b79-69a3-47ce-89a8-7be9aeda0eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:28.385824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43253b79-69a3-47ce-89a8-7be9aeda0eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:28.386325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43253b79-69a3-47ce-89a8-7be9aeda0eb1 None admin] PATH_INFO: `/v3/users/b28eecf89c8a402a963a4e1bcf0b294b` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:28.387823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43253b79-69a3-47ce-89a8-7be9aeda0eb1 None admin] RBAC: Authorizing `identity:get_user(user_id=b28eecf89c8a402a963a4e1bcf0b294b)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:28.389510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43253b79-69a3-47ce-89a8-7be9aeda0eb1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:28.395732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 135/264] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:28 2022] GET /identity/v3/users/b28eecf89c8a402a963a4e1bcf0b294b => generated 297 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:28.397541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:28.400876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43253b79-69a3-47ce-89a8-7be9aeda0eb1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:28.404448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd436c0d-2d95-4606-b877-176f38a0f687 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:28.423613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:28.439126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad1a1-15f2-4f0f-ab62-e6f7268a7436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:28.439813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad1a1-15f2-4f0f-ab62-e6f7268a7436 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:28.440093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad1a1-15f2-4f0f-ab62-e6f7268a7436 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:28.440368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bad1a1-15f2-4f0f-ab62-e6f7268a7436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:28.508396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bad1a1-15f2-4f0f-ab62-e6f7268a7436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:28.509885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bad1a1-15f2-4f0f-ab62-e6f7268a7436 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:28.615144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:28.619503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 130/265] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:28.620376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad1a1-15f2-4f0f-ab62-e6f7268a7436 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:28.717347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43253b79-69a3-47ce-89a8-7be9aeda0eb1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:28.728318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bad1a1-15f2-4f0f-ab62-e6f7268a7436 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:28.767660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2a488-7c4d-4565-8de1-b88693341f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:28.769227 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2a488-7c4d-4565-8de1-b88693341f9e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:28.769719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2a488-7c4d-4565-8de1-b88693341f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:28.770181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2a488-7c4d-4565-8de1-b88693341f9e None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:28.771610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2a488-7c4d-4565-8de1-b88693341f9e None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:28.773258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2a488-7c4d-4565-8de1-b88693341f9e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:28.776129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 131/266] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:28 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:28.781467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2a488-7c4d-4565-8de1-b88693341f9e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:28.850474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:28.852063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c2bbb-5109-454b-8b69-3ca406e63b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:28.852981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2bbb-5109-454b-8b69-3ca406e63b7e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:28.853269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2bbb-5109-454b-8b69-3ca406e63b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:28.853543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c2bbb-5109-454b-8b69-3ca406e63b7e None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:28.860869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c2bbb-5109-454b-8b69-3ca406e63b7e None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:28.861797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06c2bbb-5109-454b-8b69-3ca406e63b7e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:28.863649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 136/267] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:28 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 463 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:28.870060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c2bbb-5109-454b-8b69-3ca406e63b7e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:28.894463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2a488-7c4d-4565-8de1-b88693341f9e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:28.956668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfc5b4-9011-4c57-9866-9465f6deb0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:28.958280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc5b4-9011-4c57-9866-9465f6deb0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:28.963969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc5b4-9011-4c57-9866-9465f6deb0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:28.964475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfc5b4-9011-4c57-9866-9465f6deb0a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:28.965527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfc5b4-9011-4c57-9866-9465f6deb0a0 None admin] RBAC: Adding query filter params (role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=efcdcc966ddd493f83f40b196e076f1a, user.id=cf16d55421fb44f99dedfb8d0d28ced6) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:28.966029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfc5b4-9011-4c57-9866-9465f6deb0a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:28.967865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bfc5b4-9011-4c57-9866-9465f6deb0a0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:28.979251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 132/268] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:28 2022] GET /identity/v3/role_assignments?user.id=cf16d55421fb44f99dedfb8d0d28ced6&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:29.015587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:29.037499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188ce3-0755-4582-99b9-b86bbc9abbd5 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.038126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188ce3-0755-4582-99b9-b86bbc9abbd5 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.038752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2188ce3-0755-4582-99b9-b86bbc9abbd5 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.041751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 133/269] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:29.044135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c2bbb-5109-454b-8b69-3ca406e63b7e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:29.053459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8850-565a-4258-a908-e864afb07398 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.054085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8850-565a-4258-a908-e864afb07398 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.054549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c8850-565a-4258-a908-e864afb07398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.074765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-711c8850-565a-4258-a908-e864afb07398 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:29.076833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-711c8850-565a-4258-a908-e864afb07398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:29.129546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64ae11-50a8-476f-a716-c75f6daa11ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:29.130471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64ae11-50a8-476f-a716-c75f6daa11ce None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.130754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64ae11-50a8-476f-a716-c75f6daa11ce None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.131027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64ae11-50a8-476f-a716-c75f6daa11ce None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a/users/b28eecf89c8a402a963a4e1bcf0b294b/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.137073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64ae11-50a8-476f-a716-c75f6daa11ce None admin] RBAC: Authorizing `identity:create_grant(project_id=efcdcc966ddd493f83f40b196e076f1a, user_id=b28eecf89c8a402a963a4e1bcf0b294b, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:29.139114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64ae11-50a8-476f-a716-c75f6daa11ce None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:29.160624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 137/270] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:28 2022] PUT /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a/users/b28eecf89c8a402a963a4e1bcf0b294b/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:29.162400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49148334-3445-4736-a5e1-dfabeb964c2c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.162780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49148334-3445-4736-a5e1-dfabeb964c2c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.163064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49148334-3445-4736-a5e1-dfabeb964c2c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.165177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 134/272] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:29.173504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 138/272] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:29.173711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e342a8-47c6-4aa7-88de-e87138f73303 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.173711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e342a8-47c6-4aa7-88de-e87138f73303 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.173711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e342a8-47c6-4aa7-88de-e87138f73303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.196110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f3e342a8-47c6-4aa7-88de-e87138f73303 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:29.197433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f3e342a8-47c6-4aa7-88de-e87138f73303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:29.210843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49148334-3445-4736-a5e1-dfabeb964c2c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:29.263974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:29.283915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 135/273] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:29.321285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e342a8-47c6-4aa7-88de-e87138f73303 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:29.375990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49148334-3445-4736-a5e1-dfabeb964c2c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:29.432539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38d6df-4db2-4f29-9b85-58914f616067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:29.432931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38d6df-4db2-4f29-9b85-58914f616067 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.433209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38d6df-4db2-4f29-9b85-58914f616067 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.433479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc38d6df-4db2-4f29-9b85-58914f616067 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.437232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:29.447559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc38d6df-4db2-4f29-9b85-58914f616067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-bc38d6df-4db2-4f29-9b85-58914f616067 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:29.455540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:29.456410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:29.457176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:29.457772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e342a8-47c6-4aa7-88de-e87138f73303 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:29.460021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 139/274] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:29.468255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38d6df-4db2-4f29-9b85-58914f616067 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:29.496413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a94aa4-f95e-417e-a7ae-0c9fb3472765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:29.497596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94aa4-f95e-417e-a7ae-0c9fb3472765 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.498072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94aa4-f95e-417e-a7ae-0c9fb3472765 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.498565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a94aa4-f95e-417e-a7ae-0c9fb3472765 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.500156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a94aa4-f95e-417e-a7ae-0c9fb3472765 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:29.501791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a94aa4-f95e-417e-a7ae-0c9fb3472765 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:29.504169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 136/275] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:29.509123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a94aa4-f95e-417e-a7ae-0c9fb3472765 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:29.577686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc38d6df-4db2-4f29-9b85-58914f616067 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:29.584222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:29.621930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:29.623077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.623833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.624270 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.625426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:29.626020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:29.629817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:29.632270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a94aa4-f95e-417e-a7ae-0c9fb3472765 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:29.646284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 140/276] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:29.647507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:29.672901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb27e1-38c7-4404-beac-42cec7b57058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:29.673577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb27e1-38c7-4404-beac-42cec7b57058 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.674090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb27e1-38c7-4404-beac-42cec7b57058 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.674371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bb27e1-38c7-4404-beac-42cec7b57058 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.675093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb27e1-38c7-4404-beac-42cec7b57058 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:29.677261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bb27e1-38c7-4404-beac-42cec7b57058 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:29.707351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 137/277] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:29 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:29.721513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:29.776036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45800e92-ac71-4305-bec2-535261daf7c9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:29.814826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce12a9-158e-4210-b317-95eb366a3e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:29.815760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce12a9-158e-4210-b317-95eb366a3e08 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.816045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce12a9-158e-4210-b317-95eb366a3e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.816316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce12a9-158e-4210-b317-95eb366a3e08 None admin] PATH_INFO: `/v3/users/115ad77ce2d0454f8e93302d2bd8f810` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.832608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce12a9-158e-4210-b317-95eb366a3e08 None admin] RBAC: Authorizing `identity:get_user(user_id=115ad77ce2d0454f8e93302d2bd8f810)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:29.834100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce12a9-158e-4210-b317-95eb366a3e08 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:29.836116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 141/278] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:29 2022] GET /identity/v3/users/115ad77ce2d0454f8e93302d2bd8f810 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:29.842098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb27e1-38c7-4404-beac-42cec7b57058 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:29.948783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bb27e1-38c7-4404-beac-42cec7b57058 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:29.962977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:29.987040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06ba2c-e8a0-4849-a0b5-4086bd4ad2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:29.988134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06ba2c-e8a0-4849-a0b5-4086bd4ad2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:29.988613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06ba2c-e8a0-4849-a0b5-4086bd4ad2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:29.989107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06ba2c-e8a0-4849-a0b5-4086bd4ad2c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:29.990144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06ba2c-e8a0-4849-a0b5-4086bd4ad2c5 None admin] RBAC: Adding query filter params (role.id=580338b3f2b847c78aef559dbaaf51e4, user.id=115ad77ce2d0454f8e93302d2bd8f810, scope.system=all) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:29.990679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06ba2c-e8a0-4849-a0b5-4086bd4ad2c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:29.992425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06ba2c-e8a0-4849-a0b5-4086bd4ad2c5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:29.998296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 138/279] 10.222.0.69 () {56 vars in 1308 bytes} [Fri May 20 08:59:29 2022] GET /identity/v3/role_assignments?user.id=115ad77ce2d0454f8e93302d2bd8f810&scope.system=all&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:30.244006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864f9d-8d0d-40c9-a82f-dc3049206b08 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:30.244184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864f9d-8d0d-40c9-a82f-dc3049206b08 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:30.244454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf864f9d-8d0d-40c9-a82f-dc3049206b08 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:30.245990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 142/280] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:30.251376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a81ee-542d-48fa-a4ff-f37453343e81 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:30.251890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a81ee-542d-48fa-a4ff-f37453343e81 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:30.252354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a81ee-542d-48fa-a4ff-f37453343e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:30.269919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-c08a81ee-542d-48fa-a4ff-f37453343e81 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:30.271445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c08a81ee-542d-48fa-a4ff-f37453343e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:30.352723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 139/281] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:30.392855 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864f9d-8d0d-40c9-a82f-dc3049206b08 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:30.536893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf864f9d-8d0d-40c9-a82f-dc3049206b08 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:30.560629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:30.585168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7c490-deea-4c24-8e3f-4fcd75f6f5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:30.586019 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7c490-deea-4c24-8e3f-4fcd75f6f5dc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:30.586305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7c490-deea-4c24-8e3f-4fcd75f6f5dc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:30.586578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f7c490-deea-4c24-8e3f-4fcd75f6f5dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:30.613139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f7c490-deea-4c24-8e3f-4fcd75f6f5dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-41f7c490-deea-4c24-8e3f-4fcd75f6f5dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:30.620989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:30.621880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:30.622684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:30.628589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 143/282] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:30.640404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a81ee-542d-48fa-a4ff-f37453343e81 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:30.754613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a81ee-542d-48fa-a4ff-f37453343e81 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:30.764671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:30.801468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:30.802052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:30.802465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:30.802581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:30.803379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:30.803684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:30.805131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:30.822164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 140/283] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:30.828902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7c490-deea-4c24-8e3f-4fcd75f6f5dc None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:30.993068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f7c490-deea-4c24-8e3f-4fcd75f6f5dc None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:31.042505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b49e1b-58c5-4a01-b4cb-4f3612f139e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:31.043421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b49e1b-58c5-4a01-b4cb-4f3612f139e9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:31.043700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b49e1b-58c5-4a01-b4cb-4f3612f139e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:31.043970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b49e1b-58c5-4a01-b4cb-4f3612f139e9 None admin] PATH_INFO: `/v3/users/c340785b17254d15887d496fef472f90` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:31.045157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b49e1b-58c5-4a01-b4cb-4f3612f139e9 None admin] RBAC: Authorizing `identity:get_user(user_id=c340785b17254d15887d496fef472f90)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:31.047190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b49e1b-58c5-4a01-b4cb-4f3612f139e9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:31.049846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 144/284] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:30 2022] GET /identity/v3/users/c340785b17254d15887d496fef472f90 => generated 297 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:31.054286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:31.203031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2021e608-6352-442b-af0a-6ec0054863b3 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:31.251982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbcca98-6c64-478b-88ba-5ac3b83f74c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:31.252871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcca98-6c64-478b-88ba-5ac3b83f74c7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:31.253152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcca98-6c64-478b-88ba-5ac3b83f74c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:31.253425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbcca98-6c64-478b-88ba-5ac3b83f74c7 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:31.254647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcca98-6c64-478b-88ba-5ac3b83f74c7 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:31.256153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbcca98-6c64-478b-88ba-5ac3b83f74c7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:31.257922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 141/285] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:31 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:31.263806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b49e1b-58c5-4a01-b4cb-4f3612f139e9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:31.331430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:31.410077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b49e1b-58c5-4a01-b4cb-4f3612f139e9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:31.458358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d830cb-ed98-47a9-8976-2519120f3465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:31.459301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d830cb-ed98-47a9-8976-2519120f3465 None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:31.459584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d830cb-ed98-47a9-8976-2519120f3465 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:31.459852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d830cb-ed98-47a9-8976-2519120f3465 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a/users/c340785b17254d15887d496fef472f90/roles/dd10b97be2224165a8d312ca6eefdbfd` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:31.461913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d830cb-ed98-47a9-8976-2519120f3465 None admin] RBAC: Authorizing `identity:create_grant(project_id=efcdcc966ddd493f83f40b196e076f1a, user_id=c340785b17254d15887d496fef472f90, role_id=dd10b97be2224165a8d312ca6eefdbfd)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:31.463395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d830cb-ed98-47a9-8976-2519120f3465 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:31.490798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 145/286] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 08:59:31 2022] PUT /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a/users/c340785b17254d15887d496fef472f90/roles/dd10b97be2224165a8d312ca6eefdbfd => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:32.158119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2f86e-596a-4a65-ba95-56027ebf6a8e None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.158620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2f86e-596a-4a65-ba95-56027ebf6a8e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.158620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2f86e-596a-4a65-ba95-56027ebf6a8e None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.159978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 142/287] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:32.172895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29767ae2-918c-4aa7-9939-b98d332605ca None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.173415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29767ae2-918c-4aa7-9939-b98d332605ca None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.173415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29767ae2-918c-4aa7-9939-b98d332605ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.196384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-29767ae2-918c-4aa7-9939-b98d332605ca None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:32.197943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-29767ae2-918c-4aa7-9939-b98d332605ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:32.285609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 146/288] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:32.312603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2f86e-596a-4a65-ba95-56027ebf6a8e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:32.376374 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:32.464047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2f86e-596a-4a65-ba95-56027ebf6a8e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:32.507760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfff499-822f-4c6a-9254-c77bfd5df302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:32.508683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfff499-822f-4c6a-9254-c77bfd5df302 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.508961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfff499-822f-4c6a-9254-c77bfd5df302 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.509235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfff499-822f-4c6a-9254-c77bfd5df302 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.526751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfff499-822f-4c6a-9254-c77bfd5df302 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-adfff499-822f-4c6a-9254-c77bfd5df302 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:32.534766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:32.535778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:32.536556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:32.540988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 143/289] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:32.547128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29767ae2-918c-4aa7-9939-b98d332605ca None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:32.601429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:32.688317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29767ae2-918c-4aa7-9939-b98d332605ca None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:32.702886 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4ce26-2842-432f-9cee-1083b3314236 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.703188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4ce26-2842-432f-9cee-1083b3314236 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.703466 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f4ce26-2842-432f-9cee-1083b3314236 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.704970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 144/290] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:32.710726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b6fd2-db52-4297-856b-d348279e6c77 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.711071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b6fd2-db52-4297-856b-d348279e6c77 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.711371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b6fd2-db52-4297-856b-d348279e6c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.730898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-039b6fd2-db52-4297-856b-d348279e6c77 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:32.732023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-039b6fd2-db52-4297-856b-d348279e6c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:32.747999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9d812-f9d0-4be4-9589-b3fff0584dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:32.748683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9d812-f9d0-4be4-9589-b3fff0584dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.748961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9d812-f9d0-4be4-9589-b3fff0584dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.749232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9d812-f9d0-4be4-9589-b3fff0584dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.749990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9d812-f9d0-4be4-9589-b3fff0584dd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:32.750292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9d812-f9d0-4be4-9589-b3fff0584dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:32.751705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e9d812-f9d0-4be4-9589-b3fff0584dd2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:32.767704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 147/291] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:32.768757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb8ca-ad24-420d-8e42-15f2cc476156 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.771525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb8ca-ad24-420d-8e42-15f2cc476156 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.772054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bb8ca-ad24-420d-8e42-15f2cc476156 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.773908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 148/292] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:32.775277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d02f6a-68ba-425e-a935-8e04a30edb8c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.775752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d02f6a-68ba-425e-a935-8e04a30edb8c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.776210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d02f6a-68ba-425e-a935-8e04a30edb8c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.779599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 149/293] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:32.779599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d02f6a-68ba-425e-a935-8e04a30edb8c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:32.826942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 145/294] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:32.828603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdc5ed-c64f-43ee-96c3-03929ffe79b4 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.828888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdc5ed-c64f-43ee-96c3-03929ffe79b4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.829165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdc5ed-c64f-43ee-96c3-03929ffe79b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.847592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-15fdc5ed-c64f-43ee-96c3-03929ffe79b4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:32.848928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-15fdc5ed-c64f-43ee-96c3-03929ffe79b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:32.870997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:32.879861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:32.887333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d02f6a-68ba-425e-a935-8e04a30edb8c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:32.917351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 146/295] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:32.918816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace5692-dcbe-4ddb-a670-53925058a82e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.919096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace5692-dcbe-4ddb-a670-53925058a82e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.919409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eace5692-dcbe-4ddb-a670-53925058a82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.932991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7c3a9-726f-4971-96c1-bcd0570d1c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:32.934372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7c3a9-726f-4971-96c1-bcd0570d1c6b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:32.934903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7c3a9-726f-4971-96c1-bcd0570d1c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:32.935032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-eace5692-dcbe-4ddb-a670-53925058a82e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:32.935523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e7c3a9-726f-4971-96c1-bcd0570d1c6b None admin] PATH_INFO: `/v3/users/b28eecf89c8a402a963a4e1bcf0b294b` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:32.936214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-eace5692-dcbe-4ddb-a670-53925058a82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:32.937163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7c3a9-726f-4971-96c1-bcd0570d1c6b None admin] RBAC: Authorizing `identity:get_user(user_id=b28eecf89c8a402a963a4e1bcf0b294b)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:32.940065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e7c3a9-726f-4971-96c1-bcd0570d1c6b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:32.941869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 150/296] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:32 2022] GET /identity/v3/users/b28eecf89c8a402a963a4e1bcf0b294b => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:32.942518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7c3a9-726f-4971-96c1-bcd0570d1c6b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.005429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 147/297] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:33.008468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace5692-dcbe-4ddb-a670-53925058a82e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.048834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e7c3a9-726f-4971-96c1-bcd0570d1c6b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.082517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c3335-5ef5-4f6d-970c-f985f271c2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.083626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c3335-5ef5-4f6d-970c-f985f271c2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.083905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c3335-5ef5-4f6d-970c-f985f271c2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.084172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c3335-5ef5-4f6d-970c-f985f271c2c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.085328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c3335-5ef5-4f6d-970c-f985f271c2c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:33.086813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c3335-5ef5-4f6d-970c-f985f271c2c7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:33.088477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 151/298] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:33.089322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c3335-5ef5-4f6d-970c-f985f271c2c7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.110056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:33.121748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:33.127421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eace5692-dcbe-4ddb-a670-53925058a82e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.162581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d044a4-bde0-4a85-af79-8f0bfc6efdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.163398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d044a4-bde0-4a85-af79-8f0bfc6efdee None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.163679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d044a4-bde0-4a85-af79-8f0bfc6efdee None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.163950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d044a4-bde0-4a85-af79-8f0bfc6efdee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.175411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d044a4-bde0-4a85-af79-8f0bfc6efdee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-03d044a4-bde0-4a85-af79-8f0bfc6efdee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:33.183615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:33.184479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:33.185277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 148/299] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:33.185919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d044a4-bde0-4a85-af79-8f0bfc6efdee None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.193769 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c3335-5ef5-4f6d-970c-f985f271c2c7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.234376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc070-822c-4aca-a4db-7d29ac04e655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.235026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc070-822c-4aca-a4db-7d29ac04e655 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.235331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc070-822c-4aca-a4db-7d29ac04e655 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.235608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adddc070-822c-4aca-a4db-7d29ac04e655 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.295371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d044a4-bde0-4a85-af79-8f0bfc6efdee None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.307447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc070-822c-4aca-a4db-7d29ac04e655 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:33.308931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adddc070-822c-4aca-a4db-7d29ac04e655 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:33.329645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2427d7-9e26-448d-a02c-4cab7ab0e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.330838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2427d7-9e26-448d-a02c-4cab7ab0e0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.331125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2427d7-9e26-448d-a02c-4cab7ab0e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.331438 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2427d7-9e26-448d-a02c-4cab7ab0e0d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.340266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:33.342364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2427d7-9e26-448d-a02c-4cab7ab0e0d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-ed2427d7-9e26-448d-a02c-4cab7ab0e0d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:33.350272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:33.351141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:33.352007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 149/300] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:33.352651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2427d7-9e26-448d-a02c-4cab7ab0e0d4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.367277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:33.417900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 152/301] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:33.418825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc070-822c-4aca-a4db-7d29ac04e655 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.470956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2427d7-9e26-448d-a02c-4cab7ab0e0d4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.505957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801c15c-8f42-4b47-8c80-ff6678ea7202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.506624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801c15c-8f42-4b47-8c80-ff6678ea7202 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.506910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801c15c-8f42-4b47-8c80-ff6678ea7202 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.507196 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9801c15c-8f42-4b47-8c80-ff6678ea7202 None admin] PATH_INFO: `/v3/groups` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.507946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801c15c-8f42-4b47-8c80-ff6678ea7202 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:33.509408 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9801c15c-8f42-4b47-8c80-ff6678ea7202 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:33.523231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adddc070-822c-4aca-a4db-7d29ac04e655 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.526349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 150/302] 10.222.0.69 () {60 vars in 1134 bytes} [Fri May 20 08:59:33 2022] POST /identity/v3/groups => generated 229 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:33.527005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801c15c-8f42-4b47-8c80-ff6678ea7202 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.572658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:33.581227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.582088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.583321 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.583610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.584401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:33.584712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:33.586286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:33.604029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 153/303] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:33.604029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.627091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:33.678392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9801c15c-8f42-4b47-8c80-ff6678ea7202 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.725535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.726489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bed4e2-5395-4ca9-9045-77159faae24c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.726870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.727365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.727826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.728855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:33.729344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:33.730934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:33.741304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 151/304] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:33.742436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.772434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e692c5-2f36-42cd-8a6e-f128b606cb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.773435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e692c5-2f36-42cd-8a6e-f128b606cb57 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.773722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e692c5-2f36-42cd-8a6e-f128b606cb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.774657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e692c5-2f36-42cd-8a6e-f128b606cb57 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.775361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e692c5-2f36-42cd-8a6e-f128b606cb57 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:33.777492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e692c5-2f36-42cd-8a6e-f128b606cb57 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:33.778937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 154/305] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:33.779664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e692c5-2f36-42cd-8a6e-f128b606cb57 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.871107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:33.882534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65503ef9-97dc-48f3-b234-50c7ddd00982 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.912517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:33.923301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f11629-c59b-42ee-99e2-32a14a141f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.924222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f11629-c59b-42ee-99e2-32a14a141f3f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.924501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f11629-c59b-42ee-99e2-32a14a141f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.924775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f11629-c59b-42ee-99e2-32a14a141f3f None admin] PATH_INFO: `/v3/users/115ad77ce2d0454f8e93302d2bd8f810` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.926071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f11629-c59b-42ee-99e2-32a14a141f3f None admin] RBAC: Authorizing `identity:get_user(user_id=115ad77ce2d0454f8e93302d2bd8f810)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:33.927584 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f11629-c59b-42ee-99e2-32a14a141f3f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:33.929002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e692c5-2f36-42cd-8a6e-f128b606cb57 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:33.929670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 152/306] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/users/115ad77ce2d0454f8e93302d2bd8f810 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:33.930515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f11629-c59b-42ee-99e2-32a14a141f3f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:33.967183 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32609d6-0f8f-4fd2-b519-085002f8953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:33.968138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32609d6-0f8f-4fd2-b519-085002f8953b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:33.968418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32609d6-0f8f-4fd2-b519-085002f8953b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:33.968685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32609d6-0f8f-4fd2-b519-085002f8953b None admin] PATH_INFO: `/v3/users/809427c84a9e4b6d832ba8a5c05efbc6` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:33.983021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32609d6-0f8f-4fd2-b519-085002f8953b None admin] RBAC: Authorizing `identity:get_user(user_id=809427c84a9e4b6d832ba8a5c05efbc6)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:33.984527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32609d6-0f8f-4fd2-b519-085002f8953b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:33.986648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 155/307] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/users/809427c84a9e4b6d832ba8a5c05efbc6 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:33.987516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32609d6-0f8f-4fd2-b519-085002f8953b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:34.052710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f11629-c59b-42ee-99e2-32a14a141f3f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:34.086856 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:34.087554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:34.087836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:34.088106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:34.088943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] RBAC: Adding query filter params (role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=efcdcc966ddd493f83f40b196e076f1a, user.id=b28eecf89c8a402a963a4e1bcf0b294b) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:34.089319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:34.090743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:34.103600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 153/308] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:33 2022] GET /identity/v3/role_assignments?user.id=b28eecf89c8a402a963a4e1bcf0b294b&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:34.104414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:34.120424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32609d6-0f8f-4fd2-b519-085002f8953b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:34.152668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:34.174317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed00b08e-8ee1-40f2-be9e-f5b02d2f91fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:34.175638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00b08e-8ee1-40f2-be9e-f5b02d2f91fc None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:34.175990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00b08e-8ee1-40f2-be9e-f5b02d2f91fc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:34.176864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed00b08e-8ee1-40f2-be9e-f5b02d2f91fc None admin] PATH_INFO: `/v3/system/users/115ad77ce2d0454f8e93302d2bd8f810/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:34.179113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00b08e-8ee1-40f2-be9e-f5b02d2f91fc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=115ad77ce2d0454f8e93302d2bd8f810, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:34.180016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:34.182104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed00b08e-8ee1-40f2-be9e-f5b02d2f91fc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:34.197400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 156/309] 10.222.0.69 () {58 vars in 1334 bytes} [Fri May 20 08:59:33 2022] PUT /identity/v3/system/users/115ad77ce2d0454f8e93302d2bd8f810/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:34.198986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7d742-fbc1-4329-9038-3a7bcf825e52 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:34.199311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7d742-fbc1-4329-9038-3a7bcf825e52 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:34.199603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f7d742-fbc1-4329-9038-3a7bcf825e52 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:34.201383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 157/310] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:34.207921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecff0a-46fc-44a5-b1b8-63d9bac47d1f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:34.208241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecff0a-46fc-44a5-b1b8-63d9bac47d1f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:34.208487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecff0a-46fc-44a5-b1b8-63d9bac47d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:34.241353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-48ecff0a-46fc-44a5-b1b8-63d9bac47d1f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:34.242735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-48ecff0a-46fc-44a5-b1b8-63d9bac47d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:34.297642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33ffb3-247f-437f-b1f7-3d4895a827c5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:34.335142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 158/311] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:34.376448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecff0a-46fc-44a5-b1b8-63d9bac47d1f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:34.379941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:34.380786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:34.381215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:34.381624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:34.382613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] RBAC: Adding query filter params (role.id=dd10b97be2224165a8d312ca6eefdbfd, user.id=809427c84a9e4b6d832ba8a5c05efbc6, scope.system=all) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:34.383200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:34.384878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:34.391950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 154/312] 10.222.0.69 () {56 vars in 1308 bytes} [Fri May 20 08:59:34 2022] GET /identity/v3/role_assignments?user.id=809427c84a9e4b6d832ba8a5c05efbc6&scope.system=all&role.id=dd10b97be2224165a8d312ca6eefdbfd => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:34.507408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecff0a-46fc-44a5-b1b8-63d9bac47d1f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:34.523698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:34.549184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f5b87-a60e-4681-bda5-0b82f1aa4f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:34.550479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f5b87-a60e-4681-bda5-0b82f1aa4f4a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:34.550962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f5b87-a60e-4681-bda5-0b82f1aa4f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:34.551467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110f5b87-a60e-4681-bda5-0b82f1aa4f4a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:34.563511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110f5b87-a60e-4681-bda5-0b82f1aa4f4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-110f5b87-a60e-4681-bda5-0b82f1aa4f4a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:34.571966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:34.572888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:34.573705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:34.576558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 159/313] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:34.583099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:34.696845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af6be8-bba9-4a16-ac4a-08480bdfaae8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:34.727610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:34.736036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:34.737035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:34.737520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:34.737989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:34.738962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:34.739533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:34.741170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:34.751656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 155/314] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:34.760042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f5b87-a60e-4681-bda5-0b82f1aa4f4a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:34.867770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110f5b87-a60e-4681-bda5-0b82f1aa4f4a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:34.902166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb3ddc-658e-4abf-ab29-ad941b77b724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:34.903382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb3ddc-658e-4abf-ab29-ad941b77b724 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:34.903843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb3ddc-658e-4abf-ab29-ad941b77b724 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:34.904294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb3ddc-658e-4abf-ab29-ad941b77b724 None admin] PATH_INFO: `/v3/users/c340785b17254d15887d496fef472f90` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:34.905727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb3ddc-658e-4abf-ab29-ad941b77b724 None admin] RBAC: Authorizing `identity:get_user(user_id=c340785b17254d15887d496fef472f90)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:34.907404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb3ddc-658e-4abf-ab29-ad941b77b724 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:34.913866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 160/315] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:34 2022] GET /identity/v3/users/c340785b17254d15887d496fef472f90 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:34.915482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:35.017940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5029538-5a6c-4720-a443-c6de79fec39a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:35.052084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ffec6-773a-4371-bc22-df97a808cd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:35.052731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ffec6-773a-4371-bc22-df97a808cd17 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:35.053012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ffec6-773a-4371-bc22-df97a808cd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:35.053283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2ffec6-773a-4371-bc22-df97a808cd17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:35.111681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:35.120180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ffec6-773a-4371-bc22-df97a808cd17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:35.121675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2ffec6-773a-4371-bc22-df97a808cd17 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:35.220689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 156/316] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:35.226237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb3ddc-658e-4abf-ab29-ad941b77b724 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:35.269281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:35.332959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb3ddc-658e-4abf-ab29-ad941b77b724 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:35.370574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dbd6d3-de27-42cf-9b58-a23de65e2dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:35.371471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd6d3-de27-42cf-9b58-a23de65e2dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:35.371753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd6d3-de27-42cf-9b58-a23de65e2dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:35.372028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dbd6d3-de27-42cf-9b58-a23de65e2dc1 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:35.373215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbd6d3-de27-42cf-9b58-a23de65e2dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:35.374687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4dbd6d3-de27-42cf-9b58-a23de65e2dc1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:35.376713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 161/317] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:35 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:35.382232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ffec6-773a-4371-bc22-df97a808cd17 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:35.469894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:35.485091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2ffec6-773a-4371-bc22-df97a808cd17 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:35.519421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4183c0-e35f-4138-b901-0ef102dd50b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:35.520009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4183c0-e35f-4138-b901-0ef102dd50b7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:35.520282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4183c0-e35f-4138-b901-0ef102dd50b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:35.520543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4183c0-e35f-4138-b901-0ef102dd50b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:35.521375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4183c0-e35f-4138-b901-0ef102dd50b7 None admin] RBAC: Adding query filter params (role.id=dd10b97be2224165a8d312ca6eefdbfd, scope.project.id=efcdcc966ddd493f83f40b196e076f1a, user.id=c340785b17254d15887d496fef472f90) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:35.521682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4183c0-e35f-4138-b901-0ef102dd50b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:35.523207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4183c0-e35f-4138-b901-0ef102dd50b7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:35.529871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 157/318] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 08:59:35 2022] GET /identity/v3/role_assignments?user.id=c340785b17254d15887d496fef472f90&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=dd10b97be2224165a8d312ca6eefdbfd => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:35.906794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c7863-8a29-4867-8d82-b2876753b81f None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:35.907037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c7863-8a29-4867-8d82-b2876753b81f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:35.907337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c7863-8a29-4867-8d82-b2876753b81f None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:35.908727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 162/319] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:35.914369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d3b46-ee5f-4e81-9666-fad6a953487d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:35.914638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d3b46-ee5f-4e81-9666-fad6a953487d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:35.914902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d3b46-ee5f-4e81-9666-fad6a953487d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:35.932174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-9c2d3b46-ee5f-4e81-9666-fad6a953487d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:35.933359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9c2d3b46-ee5f-4e81-9666-fad6a953487d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:35.998294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 158/320] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:36.020315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c7863-8a29-4867-8d82-b2876753b81f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:36.126050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c7863-8a29-4867-8d82-b2876753b81f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:36.136950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:36.162133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3176117b-20b9-430f-9aa8-18bcda50b357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:36.162937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176117b-20b9-430f-9aa8-18bcda50b357 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:36.163219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176117b-20b9-430f-9aa8-18bcda50b357 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:36.163550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3176117b-20b9-430f-9aa8-18bcda50b357 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:36.164794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176117b-20b9-430f-9aa8-18bcda50b357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:36.166602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3176117b-20b9-430f-9aa8-18bcda50b357 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:36.168471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 163/321] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:36.173946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d3b46-ee5f-4e81-9666-fad6a953487d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:36.260649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:36.282891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d3b46-ee5f-4e81-9666-fad6a953487d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:36.318738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ff19ca-6fca-4ba7-9189-0316c37b6963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:36.319429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff19ca-6fca-4ba7-9189-0316c37b6963 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:36.319710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff19ca-6fca-4ba7-9189-0316c37b6963 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:36.319985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ff19ca-6fca-4ba7-9189-0316c37b6963 None admin] PATH_INFO: `/v3/groups` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:36.320712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff19ca-6fca-4ba7-9189-0316c37b6963 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:36.322183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ff19ca-6fca-4ba7-9189-0316c37b6963 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:36.339495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 159/322] 10.222.0.69 () {60 vars in 1134 bytes} [Fri May 20 08:59:36 2022] POST /identity/v3/groups => generated 226 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:36.600870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7ffc-cb49-4478-9bea-92b8c6e5e1f0 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:36.601396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7ffc-cb49-4478-9bea-92b8c6e5e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:36.601396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b7ffc-cb49-4478-9bea-92b8c6e5e1f0 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:36.603090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 164/323] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:36.608639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88865-3127-4643-ae30-b0afd54dfb79 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:36.608928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88865-3127-4643-ae30-b0afd54dfb79 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:36.609199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f88865-3127-4643-ae30-b0afd54dfb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:36.626735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b1f88865-3127-4643-ae30-b0afd54dfb79 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:36.628313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b1f88865-3127-4643-ae30-b0afd54dfb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:36.700434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 160/324] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:36.704018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12579e-7164-4525-9bf8-a24b8aa4bc38 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:36.704309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12579e-7164-4525-9bf8-a24b8aa4bc38 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:36.704585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12579e-7164-4525-9bf8-a24b8aa4bc38 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:36.705937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 165/325] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:36.711470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02df97c-3ad1-474b-8d46-984c7321aa6a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:36.711761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02df97c-3ad1-474b-8d46-984c7321aa6a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:36.712040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02df97c-3ad1-474b-8d46-984c7321aa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:36.726985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12579e-7164-4525-9bf8-a24b8aa4bc38 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:36.729815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d02df97c-3ad1-474b-8d46-984c7321aa6a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:36.732255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d02df97c-3ad1-474b-8d46-984c7321aa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:36.816372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:36.822509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 161/326] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:36.848144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02df97c-3ad1-474b-8d46-984c7321aa6a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:36.867631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12579e-7164-4525-9bf8-a24b8aa4bc38 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:36.909198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811eec17-de09-451d-abc5-7b473a9ee8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:36.910522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eec17-de09-451d-abc5-7b473a9ee8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:36.911103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eec17-de09-451d-abc5-7b473a9ee8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:36.911627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811eec17-de09-451d-abc5-7b473a9ee8f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:36.923727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811eec17-de09-451d-abc5-7b473a9ee8f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-811eec17-de09-451d-abc5-7b473a9ee8f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:36.932187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:36.933088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:36.933863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:36.936706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 166/327] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:36.946427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811eec17-de09-451d-abc5-7b473a9ee8f4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:36.962446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02df97c-3ad1-474b-8d46-984c7321aa6a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:37.008000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049624-d646-41f6-9a0c-5d037ddd6867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:37.009203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049624-d646-41f6-9a0c-5d037ddd6867 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:37.009326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049624-d646-41f6-9a0c-5d037ddd6867 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:37.009519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a049624-d646-41f6-9a0c-5d037ddd6867 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:37.012670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:37.026373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a049624-d646-41f6-9a0c-5d037ddd6867 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8a049624-d646-41f6-9a0c-5d037ddd6867 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:37.035959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:37.036874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:37.037641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:37.039406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 162/328] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:37.044228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049624-d646-41f6-9a0c-5d037ddd6867 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:37.092122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811eec17-de09-451d-abc5-7b473a9ee8f4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:37.132589 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:37.140568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:37.141563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:37.142053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:37.142688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:37.145421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:37.145950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:37.148685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:37.160151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 167/329] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:37.167551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:37.196766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a049624-d646-41f6-9a0c-5d037ddd6867 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:37.235311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:37.236016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:37.236299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:37.236577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:37.237353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:37.237656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:37.240240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:37.253038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 163/330] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:37.254347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:37.291066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:37.299296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f571b-fdcc-4e92-9e73-2155ffed76bf None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:37.342015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf4909-5a97-4f31-840e-9d5e8836ec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:37.343590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4909-5a97-4f31-840e-9d5e8836ec5a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:37.343884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4909-5a97-4f31-840e-9d5e8836ec5a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:37.344437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4909-5a97-4f31-840e-9d5e8836ec5a None admin] PATH_INFO: `/v3/users/115ad77ce2d0454f8e93302d2bd8f810` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:37.345920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf4909-5a97-4f31-840e-9d5e8836ec5a None admin] RBAC: Authorizing `identity:get_user(user_id=115ad77ce2d0454f8e93302d2bd8f810)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:37.348064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cf4909-5a97-4f31-840e-9d5e8836ec5a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:37.349992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 168/331] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:37 2022] GET /identity/v3/users/115ad77ce2d0454f8e93302d2bd8f810 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:37.356377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf4909-5a97-4f31-840e-9d5e8836ec5a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:37.397519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de76b8-90d2-4c6e-b849-8c314ca7f500 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:37.424628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:37.439564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb700be-cd98-46d8-97ab-5e5b644c6df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:37.440603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb700be-cd98-46d8-97ab-5e5b644c6df5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:37.440885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb700be-cd98-46d8-97ab-5e5b644c6df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:37.441156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb700be-cd98-46d8-97ab-5e5b644c6df5 None admin] PATH_INFO: `/v3/users/809427c84a9e4b6d832ba8a5c05efbc6` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:37.443512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb700be-cd98-46d8-97ab-5e5b644c6df5 None admin] RBAC: Authorizing `identity:get_user(user_id=809427c84a9e4b6d832ba8a5c05efbc6)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:37.444974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb700be-cd98-46d8-97ab-5e5b644c6df5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:37.448190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 164/332] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:37 2022] GET /identity/v3/users/809427c84a9e4b6d832ba8a5c05efbc6 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:37.453770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb700be-cd98-46d8-97ab-5e5b644c6df5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:37.491516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf4909-5a97-4f31-840e-9d5e8836ec5a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:37.532310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa75b48-b06c-401a-8a44-430b85b28f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:37.533682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa75b48-b06c-401a-8a44-430b85b28f9f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:37.534139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa75b48-b06c-401a-8a44-430b85b28f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:37.534832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa75b48-b06c-401a-8a44-430b85b28f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:37.536090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa75b48-b06c-401a-8a44-430b85b28f9f None admin] RBAC: Adding query filter params (role.id=580338b3f2b847c78aef559dbaaf51e4, user.id=115ad77ce2d0454f8e93302d2bd8f810, scope.system=all) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:37.536869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa75b48-b06c-401a-8a44-430b85b28f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:37.538857 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa75b48-b06c-401a-8a44-430b85b28f9f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:37.554409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 169/333] 10.222.0.69 () {56 vars in 1308 bytes} [Fri May 20 08:59:37 2022] GET /identity/v3/role_assignments?user.id=115ad77ce2d0454f8e93302d2bd8f810&scope.system=all&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 510 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:37.584189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb700be-cd98-46d8-97ab-5e5b644c6df5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:37.625399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7814e79-eef4-48f4-8bcd-3aba2f3bfcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:37.626282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814e79-eef4-48f4-8bcd-3aba2f3bfcb5 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:37.626562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814e79-eef4-48f4-8bcd-3aba2f3bfcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:37.626836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7814e79-eef4-48f4-8bcd-3aba2f3bfcb5 None admin] PATH_INFO: `/v3/system/users/809427c84a9e4b6d832ba8a5c05efbc6/roles/dd10b97be2224165a8d312ca6eefdbfd` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:37.628435 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7814e79-eef4-48f4-8bcd-3aba2f3bfcb5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=809427c84a9e4b6d832ba8a5c05efbc6, role_id=dd10b97be2224165a8d312ca6eefdbfd)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:37.629933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7814e79-eef4-48f4-8bcd-3aba2f3bfcb5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:37.649106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 165/334] 10.222.0.69 () {58 vars in 1334 bytes} [Fri May 20 08:59:37 2022] PUT /identity/v3/system/users/809427c84a9e4b6d832ba8a5c05efbc6/roles/dd10b97be2224165a8d312ca6eefdbfd => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 08:59:39.482350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45229-6645-4829-996d-e0d19a9b63f8 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:39.482922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45229-6645-4829-996d-e0d19a9b63f8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:39.482922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e45229-6645-4829-996d-e0d19a9b63f8 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:39.484554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 170/335] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:39.489901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c1d73-9396-44ef-9587-3b605a76d23a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:39.490378 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c1d73-9396-44ef-9587-3b605a76d23a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:39.490461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c1d73-9396-44ef-9587-3b605a76d23a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:39.514458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-205c1d73-9396-44ef-9587-3b605a76d23a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:39.515671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-205c1d73-9396-44ef-9587-3b605a76d23a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:39.578587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:39.599398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 166/336] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:39.642770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e45229-6645-4829-996d-e0d19a9b63f8 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:39.677079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:39.770736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e45229-6645-4829-996d-e0d19a9b63f8 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:39.806697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8002249-f207-4499-bf00-28bd48c25c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:39.807544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8002249-f207-4499-bf00-28bd48c25c29 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:39.807825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8002249-f207-4499-bf00-28bd48c25c29 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:39.808101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8002249-f207-4499-bf00-28bd48c25c29 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:39.821973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8002249-f207-4499-bf00-28bd48c25c29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-a8002249-f207-4499-bf00-28bd48c25c29 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:39.829802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:39.830698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:39.831553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 171/337] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:39.838317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c1d73-9396-44ef-9587-3b605a76d23a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:39.850891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea411c-1b9a-4b57-b6a0-f6485647d56b None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:39.851212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea411c-1b9a-4b57-b6a0-f6485647d56b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:39.851572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea411c-1b9a-4b57-b6a0-f6485647d56b None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:39.852944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 172/338] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:39.863786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118233fb-19bf-4275-a8c5-74dc995201bd None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:39.864060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118233fb-19bf-4275-a8c5-74dc995201bd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:39.864334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118233fb-19bf-4275-a8c5-74dc995201bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:39.895787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-118233fb-19bf-4275-a8c5-74dc995201bd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:39.896972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-118233fb-19bf-4275-a8c5-74dc995201bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:40.016098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c1d73-9396-44ef-9587-3b605a76d23a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:40.022910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 173/339] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:40.024782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e473947-65e6-430c-98dc-cbff0091127a None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.025229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e473947-65e6-430c-98dc-cbff0091127a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.025699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e473947-65e6-430c-98dc-cbff0091127a None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.027141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 174/340] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:40.036494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c71f39-73d3-41fe-991f-fc2a26d22345 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.036969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c71f39-73d3-41fe-991f-fc2a26d22345 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.037388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c71f39-73d3-41fe-991f-fc2a26d22345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.062765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:40.063739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.064174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.064599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.065500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:40.065629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-68c71f39-73d3-41fe-991f-fc2a26d22345 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:40.066064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:40.066698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-68c71f39-73d3-41fe-991f-fc2a26d22345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:40.067610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:40.080296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:40.084964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 167/341] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:40.085970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:40.142771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:40.143797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 175/342] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:40.144804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c71f39-73d3-41fe-991f-fc2a26d22345 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:40.218838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e65a23-2b48-4aec-9689-c346187f69d3 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:40.245555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c71f39-73d3-41fe-991f-fc2a26d22345 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:40.271991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab41d9-2efd-4ccc-9c45-fed7714aa258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:40.273154 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab41d9-2efd-4ccc-9c45-fed7714aa258 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.273626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab41d9-2efd-4ccc-9c45-fed7714aa258 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.274037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ab41d9-2efd-4ccc-9c45-fed7714aa258 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.281411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e439c-07ff-418d-b550-b0c384392152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:40.282330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e439c-07ff-418d-b550-b0c384392152 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.282756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e439c-07ff-418d-b550-b0c384392152 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.283167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e439c-07ff-418d-b550-b0c384392152 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.303649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:40.311889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ab41d9-2efd-4ccc-9c45-fed7714aa258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-44ab41d9-2efd-4ccc-9c45-fed7714aa258 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:40.320074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:40.321162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:40.322134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:40.328192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 168/343] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:40.329227 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab41d9-2efd-4ccc-9c45-fed7714aa258 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:40.363828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e439c-07ff-418d-b550-b0c384392152 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:40.365775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e439c-07ff-418d-b550-b0c384392152 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:40.485280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 176/344] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:40.487166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e439c-07ff-418d-b550-b0c384392152 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:40.511047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ab41d9-2efd-4ccc-9c45-fed7714aa258 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:40.521266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:40.556016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:40.567991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf28e2e-6d6d-4361-bdad-662fbedf6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:40.568990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf28e2e-6d6d-4361-bdad-662fbedf6350 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.569376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf28e2e-6d6d-4361-bdad-662fbedf6350 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.569749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf28e2e-6d6d-4361-bdad-662fbedf6350 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.587474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf28e2e-6d6d-4361-bdad-662fbedf6350 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-4cf28e2e-6d6d-4361-bdad-662fbedf6350 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:40.595292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:40.596204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:40.597268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:40.601295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 169/345] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:40.602254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf28e2e-6d6d-4361-bdad-662fbedf6350 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:40.603942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e439c-07ff-418d-b550-b0c384392152 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:40.644133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:40.645071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.645495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.645898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.646787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:40.647213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:40.649419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:40.658552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 177/346] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:40.660942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:40.791108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf28e2e-6d6d-4361-bdad-662fbedf6350 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:40.840497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5a6a6-e933-43fe-8fe8-60214c187436 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:40.849136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3bf13-f7ff-4e52-8f12-b733e2584af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:40.849469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3bf13-f7ff-4e52-8f12-b733e2584af5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.849758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3bf13-f7ff-4e52-8f12-b733e2584af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.850028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b3bf13-f7ff-4e52-8f12-b733e2584af5 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.851735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3bf13-f7ff-4e52-8f12-b733e2584af5 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:40.852756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b3bf13-f7ff-4e52-8f12-b733e2584af5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:40.854474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 170/347] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:40 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:40.855237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3bf13-f7ff-4e52-8f12-b733e2584af5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:40.856502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:40.882120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:40.883732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:40.884402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:40.885070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:40.886171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:40.886696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:40.890270 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:40.900266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 178/348] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:40.901571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:40.911763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:41.019955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6044a-beb0-4144-b361-3b583ab54e24 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:41.024800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b3bf13-f7ff-4e52-8f12-b733e2584af5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:41.067535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a81729-b5a9-4427-a81a-e623fdb8eecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:41.068826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a81729-b5a9-4427-a81a-e623fdb8eecf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.069286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a81729-b5a9-4427-a81a-e623fdb8eecf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.069724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a81729-b5a9-4427-a81a-e623fdb8eecf None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.077099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d723a5d-2a18-4a4a-b86b-793d8391e20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:41.078012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d723a5d-2a18-4a4a-b86b-793d8391e20a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.078012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d723a5d-2a18-4a4a-b86b-793d8391e20a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.078295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d723a5d-2a18-4a4a-b86b-793d8391e20a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.080978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a81729-b5a9-4427-a81a-e623fdb8eecf None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:41.082690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a81729-b5a9-4427-a81a-e623fdb8eecf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:41.085024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 179/349] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:40 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:41.086301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a81729-b5a9-4427-a81a-e623fdb8eecf None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:41.135698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:41.171828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d723a5d-2a18-4a4a-b86b-793d8391e20a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:41.173294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d723a5d-2a18-4a4a-b86b-793d8391e20a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:41.210267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a81729-b5a9-4427-a81a-e623fdb8eecf None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:41.214440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:41.254900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f765cb-ad55-4e8d-a6c5-d1ba102c963b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:41.255605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f765cb-ad55-4e8d-a6c5-d1ba102c963b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.255782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f765cb-ad55-4e8d-a6c5-d1ba102c963b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.256058 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f765cb-ad55-4e8d-a6c5-d1ba102c963b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.309744 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 171/350] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:41.311302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d723a5d-2a18-4a4a-b86b-793d8391e20a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:41.350762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f765cb-ad55-4e8d-a6c5-d1ba102c963b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:41.352440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f765cb-ad55-4e8d-a6c5-d1ba102c963b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:41.392802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:41.445439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d723a5d-2a18-4a4a-b86b-793d8391e20a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:41.476677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 180/351] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:41.478045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19ab08-4290-493a-97db-b75d7a3f40e4 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.478331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19ab08-4290-493a-97db-b75d7a3f40e4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.478609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b19ab08-4290-493a-97db-b75d7a3f40e4 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.480000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 181/352] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:41.481050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d691fe-e2fb-4415-8772-ee0e6125233f None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.481333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d691fe-e2fb-4415-8772-ee0e6125233f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.481592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d691fe-e2fb-4415-8772-ee0e6125233f None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.482997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 182/353] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:41.483646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d691fe-e2fb-4415-8772-ee0e6125233f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:41.489673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c145f9-d7be-4888-8efd-4d8e57264f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:41.490696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c145f9-d7be-4888-8efd-4d8e57264f9e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.490902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c145f9-d7be-4888-8efd-4d8e57264f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.491251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c145f9-d7be-4888-8efd-4d8e57264f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.492379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c145f9-d7be-4888-8efd-4d8e57264f9e None admin] RBAC: Adding query filter params (group.id=7560290c75b64695bdbbbe9694fc3813, role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:41.493234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c145f9-d7be-4888-8efd-4d8e57264f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:41.495158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c145f9-d7be-4888-8efd-4d8e57264f9e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:41.499550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:41.504405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 172/354] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:41 2022] GET /identity/v3/role_assignments?group.id=7560290c75b64695bdbbbe9694fc3813&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:41.505888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f633c-187d-44ba-970c-67c5026a8e63 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.506265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f633c-187d-44ba-970c-67c5026a8e63 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.506644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f633c-187d-44ba-970c-67c5026a8e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.526976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-9d5f633c-187d-44ba-970c-67c5026a8e63 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:41.528378 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9d5f633c-187d-44ba-970c-67c5026a8e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:41.612423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d691fe-e2fb-4415-8772-ee0e6125233f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:41.622761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 173/355] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:41.622761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f633c-187d-44ba-970c-67c5026a8e63 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:41.668242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40992c48-b466-45f4-a6b7-78eaf3d1e86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:41.669712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992c48-b466-45f4-a6b7-78eaf3d1e86a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.670089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992c48-b466-45f4-a6b7-78eaf3d1e86a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.670455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40992c48-b466-45f4-a6b7-78eaf3d1e86a None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.672285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40992c48-b466-45f4-a6b7-78eaf3d1e86a None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:41.673964 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40992c48-b466-45f4-a6b7-78eaf3d1e86a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:41.676120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 183/356] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:41 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:41.677486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b0aee-48ff-4cb6-8dc7-c234ffb52a82 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.677864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b0aee-48ff-4cb6-8dc7-c234ffb52a82 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.678234 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b0aee-48ff-4cb6-8dc7-c234ffb52a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.683795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:41.698205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3c7b0aee-48ff-4cb6-8dc7-c234ffb52a82 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:41.699600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3c7b0aee-48ff-4cb6-8dc7-c234ffb52a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:41.756428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f633c-187d-44ba-970c-67c5026a8e63 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:41.779201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 184/357] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:41.787473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac419-6dc0-4f3d-8996-979041e1a3ae None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.787473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac419-6dc0-4f3d-8996-979041e1a3ae None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.787473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ac419-6dc0-4f3d-8996-979041e1a3ae None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.787473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 185/358] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:41.787473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ac419-6dc0-4f3d-8996-979041e1a3ae None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:41.799896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:41.809184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07998137-3ce0-4d2d-be85-e887ca84f46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:41.810194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07998137-3ce0-4d2d-be85-e887ca84f46d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.810654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07998137-3ce0-4d2d-be85-e887ca84f46d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.811021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07998137-3ce0-4d2d-be85-e887ca84f46d None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.812382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07998137-3ce0-4d2d-be85-e887ca84f46d None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:41.814033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07998137-3ce0-4d2d-be85-e887ca84f46d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:41.816671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 174/359] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:41 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:41.817843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07998137-3ce0-4d2d-be85-e887ca84f46d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:41.911296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ac419-6dc0-4f3d-8996-979041e1a3ae None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:41.941806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07998137-3ce0-4d2d-be85-e887ca84f46d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:41.954456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:41.961626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb644f-469d-4b56-b914-949ab71b8378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:41.962427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb644f-469d-4b56-b914-949ab71b8378 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.962703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb644f-469d-4b56-b914-949ab71b8378 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.962972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb644f-469d-4b56-b914-949ab71b8378 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.979819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb644f-469d-4b56-b914-949ab71b8378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:41.984859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8c4a-0878-4c85-9a6a-e003bb61f52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:41.985733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8c4a-0878-4c85-9a6a-e003bb61f52b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.986037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8c4a-0878-4c85-9a6a-e003bb61f52b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.986293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd8c4a-0878-4c85-9a6a-e003bb61f52b None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:41.987518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd8c4a-0878-4c85-9a6a-e003bb61f52b None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-bbfb644f-469d-4b56-b914-949ab71b8378 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:41.988517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:41.989365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:41.990145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:41.990806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd8c4a-0878-4c85-9a6a-e003bb61f52b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:41.990874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 186/360] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:41.991432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 175/361] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:41 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:41.991527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501095c-ecbb-405e-8c74-0c90910a2d92 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:41.991688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501095c-ecbb-405e-8c74-0c90910a2d92 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:41.991838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8c4a-0878-4c85-9a6a-e003bb61f52b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:41.991931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9501095c-ecbb-405e-8c74-0c90910a2d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.009672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9501095c-ecbb-405e-8c74-0c90910a2d92 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:42.010872 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9501095c-ecbb-405e-8c74-0c90910a2d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:42.075613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:42.094687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 187/362] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:42.095628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501095c-ecbb-405e-8c74-0c90910a2d92 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:42.126988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd8c4a-0878-4c85-9a6a-e003bb61f52b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.166352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb901f1-e301-4578-82e9-6b2962a636ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.167167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb901f1-e301-4578-82e9-6b2962a636ad None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.167469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb901f1-e301-4578-82e9-6b2962a636ad None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.167740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb901f1-e301-4578-82e9-6b2962a636ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.178258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb901f1-e301-4578-82e9-6b2962a636ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-6bb901f1-e301-4578-82e9-6b2962a636ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:42.186294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:42.187185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:42.188027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 176/363] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:42.188697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb901f1-e301-4578-82e9-6b2962a636ad None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:42.208623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9501095c-ecbb-405e-8c74-0c90910a2d92 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.235135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:42.252145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81838e79-52e2-4e2a-82d4-76a3b42a4985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.253841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838e79-52e2-4e2a-82d4-76a3b42a4985 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.253841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838e79-52e2-4e2a-82d4-76a3b42a4985 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.253841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838e79-52e2-4e2a-82d4-76a3b42a4985 None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.255417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81838e79-52e2-4e2a-82d4-76a3b42a4985 None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:42.256336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81838e79-52e2-4e2a-82d4-76a3b42a4985 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:42.258437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 188/364] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:42.259218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81838e79-52e2-4e2a-82d4-76a3b42a4985 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:42.318432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb901f1-e301-4578-82e9-6b2962a636ad None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.352189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:42.368686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.369395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.369681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.370003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.370778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:42.371089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:42.372521 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:42.382562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 177/365] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:42.383593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:42.389757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81838e79-52e2-4e2a-82d4-76a3b42a4985 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.441015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.441682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.442273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.442273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.443134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] RBAC: Adding query filter params (group.id=7560290c75b64695bdbbbe9694fc3813, role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:42.443413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:42.444923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:42.452375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 189/366] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/role_assignments?group.id=7560290c75b64695bdbbbe9694fc3813&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:42.453524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:42.519560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a25f92-cb95-4dcd-84e5-c15987136d20 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.531026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:42.561490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fdcd3-70ac-4ca0-8ef6-0e029579a030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.562310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fdcd3-70ac-4ca0-8ef6-0e029579a030 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.562593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fdcd3-70ac-4ca0-8ef6-0e029579a030 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.562866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fdcd3-70ac-4ca0-8ef6-0e029579a030 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.575367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735fdcd3-70ac-4ca0-8ef6-0e029579a030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-735fdcd3-70ac-4ca0-8ef6-0e029579a030 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:42.585111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:42.590155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 178/367] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:42.591217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fdcd3-70ac-4ca0-8ef6-0e029579a030 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:42.605177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0297bf3-a964-4a3c-be29-9fc825025a17 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.649563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:42.665874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.666579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.666839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.667156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.667996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:42.668303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:42.670170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:42.687018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 190/368] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:42.689890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:42.730253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fdcd3-70ac-4ca0-8ef6-0e029579a030 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.770579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.771223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.771551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.771831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.774483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] RBAC: Adding query filter params (group.id=7560290c75b64695bdbbbe9694fc3813, role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=efcdcc966ddd493f83f40b196e076f1a) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:42.774483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:42.774483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:42.782958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 179/369] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/role_assignments?group.id=7560290c75b64695bdbbbe9694fc3813&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:42.784079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:42.901719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d100ee36-42b4-4054-a63f-0a5be19f684b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.938040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5eb0e-d017-4b91-9ec3-ce274ae06c9b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:42.940652 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:42.950550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:42.983082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.983821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.984118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.984396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.985154 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:42.985464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:42.987129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:42.993967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae555784-c6a0-438c-beb1-cf03b09c071b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:42.994710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae555784-c6a0-438c-beb1-cf03b09c071b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:42.995001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae555784-c6a0-438c-beb1-cf03b09c071b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:42.995297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae555784-c6a0-438c-beb1-cf03b09c071b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:42.998406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 180/370] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:43.004055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:43.109385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae555784-c6a0-438c-beb1-cf03b09c071b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:43.110947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae555784-c6a0-438c-beb1-cf03b09c071b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:43.129417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f74720f-3d5b-4ef4-a6a1-abd487eec25f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:43.173476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e89061-e471-46ec-b064-be09b7997f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:43.174684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e89061-e471-46ec-b064-be09b7997f52 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:43.175145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e89061-e471-46ec-b064-be09b7997f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:43.175969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e89061-e471-46ec-b064-be09b7997f52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:43.226773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 191/371] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:43.227708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae555784-c6a0-438c-beb1-cf03b09c071b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:43.231560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:43.248821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:43.269851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e89061-e471-46ec-b064-be09b7997f52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:43.271690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e89061-e471-46ec-b064-be09b7997f52 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:43.338725 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae555784-c6a0-438c-beb1-cf03b09c071b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:43.377032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4d576-a7be-4fb7-b9f6-8ab29b26c195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:43.379091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d576-a7be-4fb7-b9f6-8ab29b26c195 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:43.379437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d576-a7be-4fb7-b9f6-8ab29b26c195 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:43.379728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 181/372] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:43.380089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb4d576-a7be-4fb7-b9f6-8ab29b26c195 None admin] PATH_INFO: `/v3/users/809427c84a9e4b6d832ba8a5c05efbc6` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:43.381008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e89061-e471-46ec-b064-be09b7997f52 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:43.381381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4d576-a7be-4fb7-b9f6-8ab29b26c195 None admin] RBAC: Authorizing `identity:get_user(user_id=809427c84a9e4b6d832ba8a5c05efbc6)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:43.382928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb4d576-a7be-4fb7-b9f6-8ab29b26c195 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:43.385640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 192/373] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 08:59:43 2022] GET /identity/v3/users/809427c84a9e4b6d832ba8a5c05efbc6 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:43.385977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4d576-a7be-4fb7-b9f6-8ab29b26c195 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:43.456409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:43.480805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:43.489329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e89061-e471-46ec-b064-be09b7997f52 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:43.498431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb4d576-a7be-4fb7-b9f6-8ab29b26c195 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:43.524456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ae37c-8afa-4e59-8e2e-d51379cfc65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:43.525702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ae37c-8afa-4e59-8e2e-d51379cfc65c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:43.526181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ae37c-8afa-4e59-8e2e-d51379cfc65c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:43.526649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ae37c-8afa-4e59-8e2e-d51379cfc65c None admin] PATH_INFO: `/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:43.528079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ae37c-8afa-4e59-8e2e-d51379cfc65c None admin] RBAC: Authorizing `identity:get_project(project_id=aac440dc4ce14fa1a6d365f33d6d214c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:43.529731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ae37c-8afa-4e59-8e2e-d51379cfc65c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:43.532096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 182/374] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:43 2022] GET /identity/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:43.533112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ae37c-8afa-4e59-8e2e-d51379cfc65c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:43.536118 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec312d1-3334-47a2-a662-4413a6d5de58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:43.537961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec312d1-3334-47a2-a662-4413a6d5de58 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:43.537961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec312d1-3334-47a2-a662-4413a6d5de58 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:43.537961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec312d1-3334-47a2-a662-4413a6d5de58 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:43.539223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec312d1-3334-47a2-a662-4413a6d5de58 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:43.541121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec312d1-3334-47a2-a662-4413a6d5de58 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:43.543139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 193/375] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:43 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:43.544456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec312d1-3334-47a2-a662-4413a6d5de58 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:43.661650 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ae37c-8afa-4e59-8e2e-d51379cfc65c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:43.697577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec312d1-3334-47a2-a662-4413a6d5de58 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:43.708026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:43.711280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:43.711280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:43.711280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:43.711280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] RBAC: Adding query filter params (role.id=dd10b97be2224165a8d312ca6eefdbfd, user.id=809427c84a9e4b6d832ba8a5c05efbc6, scope.system=all) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:43.711280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:43.712416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:43.724726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:43.728899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 183/376] 10.222.0.69 () {56 vars in 1308 bytes} [Fri May 20 08:59:43 2022] GET /identity/v3/role_assignments?user.id=809427c84a9e4b6d832ba8a5c05efbc6&scope.system=all&role.id=dd10b97be2224165a8d312ca6eefdbfd => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:43.729724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:43.789522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279aa03-b287-4730-afd5-51ba643463ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:43.791174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279aa03-b287-4730-afd5-51ba643463ba None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:43.791703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279aa03-b287-4730-afd5-51ba643463ba None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:43.792158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6279aa03-b287-4730-afd5-51ba643463ba None admin] PATH_INFO: `/v3/groups/146a740b8dc44628b2f9f26c860f4a82` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:43.809959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279aa03-b287-4730-afd5-51ba643463ba None admin] RBAC: Authorizing `identity:get_group(group_id=146a740b8dc44628b2f9f26c860f4a82)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:43.811810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6279aa03-b287-4730-afd5-51ba643463ba None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:43.814104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 194/377] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:43 2022] GET /identity/v3/groups/146a740b8dc44628b2f9f26c860f4a82 => generated 229 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:43.819926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279aa03-b287-4730-afd5-51ba643463ba None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:43.871654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:43.900768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed79bb-dcc5-4396-a61f-dcdeb55bca16 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:43.975098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee01ccf-064c-4ab4-aff4-0bda4257aa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:43.976032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee01ccf-064c-4ab4-aff4-0bda4257aa76 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:43.976393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee01ccf-064c-4ab4-aff4-0bda4257aa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:43.976673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee01ccf-064c-4ab4-aff4-0bda4257aa76 None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:43.979917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee01ccf-064c-4ab4-aff4-0bda4257aa76 None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:43.981400 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee01ccf-064c-4ab4-aff4-0bda4257aa76 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:43.983381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 184/378] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:43 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:44.000249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee01ccf-064c-4ab4-aff4-0bda4257aa76 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:44.041974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6279aa03-b287-4730-afd5-51ba643463ba None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:44.077308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:44.112470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee01ccf-064c-4ab4-aff4-0bda4257aa76 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:44.148669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bb933-54e8-4fa0-abfb-0c975852af44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:44.149394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb933-54e8-4fa0-abfb-0c975852af44 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:44.149680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb933-54e8-4fa0-abfb-0c975852af44 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:44.149959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bb933-54e8-4fa0-abfb-0c975852af44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:44.150854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bb933-54e8-4fa0-abfb-0c975852af44 None admin] RBAC: Adding query filter params (group.id=7560290c75b64695bdbbbe9694fc3813, role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=efcdcc966ddd493f83f40b196e076f1a) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:44.151170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bb933-54e8-4fa0-abfb-0c975852af44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:44.152649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92bb933-54e8-4fa0-abfb-0c975852af44 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:44.158927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 185/379] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:43 2022] GET /identity/v3/role_assignments?group.id=7560290c75b64695bdbbbe9694fc3813&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:44.168644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd418d5f-938d-4160-ad4a-a1136a4e01ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:44.169337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418d5f-938d-4160-ad4a-a1136a4e01ab None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:44.169621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418d5f-938d-4160-ad4a-a1136a4e01ab None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:44.169899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd418d5f-938d-4160-ad4a-a1136a4e01ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:44.170759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd418d5f-938d-4160-ad4a-a1136a4e01ab None admin] RBAC: Adding query filter params (group.id=146a740b8dc44628b2f9f26c860f4a82, role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=aac440dc4ce14fa1a6d365f33d6d214c) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:44.171073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd418d5f-938d-4160-ad4a-a1136a4e01ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:44.172569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd418d5f-938d-4160-ad4a-a1136a4e01ab None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:44.185852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 195/380] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:43 2022] GET /identity/v3/role_assignments?group.id=146a740b8dc44628b2f9f26c860f4a82&scope.project.id=aac440dc4ce14fa1a6d365f33d6d214c&role.id=6dca675ade814980a252f800fb4fa3ad => generated 260 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:44.453932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c1c9b-4549-49fd-a14b-9bfa1d51249f None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:44.454304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c1c9b-4549-49fd-a14b-9bfa1d51249f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:44.454533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c1c9b-4549-49fd-a14b-9bfa1d51249f None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:44.456240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 186/381] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:44.464067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22803e-afee-42ca-88b8-00e98aba4950 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:44.464601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22803e-afee-42ca-88b8-00e98aba4950 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:44.465101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d22803e-afee-42ca-88b8-00e98aba4950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:44.487795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3d22803e-afee-42ca-88b8-00e98aba4950 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:44.490283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3d22803e-afee-42ca-88b8-00e98aba4950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:44.573180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 196/382] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:44.601661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c1c9b-4549-49fd-a14b-9bfa1d51249f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:44.729934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c1c9b-4549-49fd-a14b-9bfa1d51249f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:44.763765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:44.780420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb570df-0c6b-4568-a83b-e38bb28cd422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:44.781507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb570df-0c6b-4568-a83b-e38bb28cd422 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:44.782010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb570df-0c6b-4568-a83b-e38bb28cd422 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:44.782285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb570df-0c6b-4568-a83b-e38bb28cd422 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:44.800918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb570df-0c6b-4568-a83b-e38bb28cd422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-bcb570df-0c6b-4568-a83b-e38bb28cd422 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:44.808660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:44.809612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:44.810414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 187/383] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:44.817740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22803e-afee-42ca-88b8-00e98aba4950 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:44.831437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:44.952470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d22803e-afee-42ca-88b8-00e98aba4950 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:44.987106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:44.987742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:44.988213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:44.988650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:44.989556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:44.990006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:44.991605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:45.000950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 197/384] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:45.006687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb570df-0c6b-4568-a83b-e38bb28cd422 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:45.138368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb570df-0c6b-4568-a83b-e38bb28cd422 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:45.179075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7dd5c-c2e1-4249-aaf8-43924c218032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:45.180310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7dd5c-c2e1-4249-aaf8-43924c218032 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:45.180761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7dd5c-c2e1-4249-aaf8-43924c218032 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:45.181197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7dd5c-c2e1-4249-aaf8-43924c218032 None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:45.186441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7dd5c-c2e1-4249-aaf8-43924c218032 None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:45.188101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7dd5c-c2e1-4249-aaf8-43924c218032 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:45.190312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 188/385] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:45 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:45.196131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:45.292368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:45.306171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05af1-0dfb-4681-9dbd-6c57e3f3a454 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:45.306907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05af1-0dfb-4681-9dbd-6c57e3f3a454 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:45.307427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05af1-0dfb-4681-9dbd-6c57e3f3a454 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:45.311535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 189/386] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:45.316798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd31cf-c11d-4294-af93-009bcacec938 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:45.317368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd31cf-c11d-4294-af93-009bcacec938 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:45.317837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cd31cf-c11d-4294-af93-009bcacec938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:45.329773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c058e-abe7-45a9-98f0-bb89545b1b78 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:45.349480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-36cd31cf-c11d-4294-af93-009bcacec938 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:45.350990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-36cd31cf-c11d-4294-af93-009bcacec938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:45.385419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d309a5c-87ef-4d3c-80a3-e4c0db9f9b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:45.387768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d309a5c-87ef-4d3c-80a3-e4c0db9f9b62 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:45.388479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d309a5c-87ef-4d3c-80a3-e4c0db9f9b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:45.389761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d309a5c-87ef-4d3c-80a3-e4c0db9f9b62 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:45.393786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d309a5c-87ef-4d3c-80a3-e4c0db9f9b62 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:45.395737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d309a5c-87ef-4d3c-80a3-e4c0db9f9b62 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:45.398283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 198/387] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:45 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:45.404140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d309a5c-87ef-4d3c-80a3-e4c0db9f9b62 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:45.463747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:45.493609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 190/388] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:45.526614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd31cf-c11d-4294-af93-009bcacec938 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:45.589486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d309a5c-87ef-4d3c-80a3-e4c0db9f9b62 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:45.651978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cd31cf-c11d-4294-af93-009bcacec938 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:45.662009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd99679-f4f5-4d27-bbb2-a7099799ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:45.662921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd99679-f4f5-4d27-bbb2-a7099799ba7e None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:45.663203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd99679-f4f5-4d27-bbb2-a7099799ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:45.663501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd99679-f4f5-4d27-bbb2-a7099799ba7e None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/groups/7560290c75b64695bdbbbe9694fc3813/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:45.667544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd99679-f4f5-4d27-bbb2-a7099799ba7e None admin] RBAC: Authorizing `identity:create_grant(project_id=0d512da984d845ada1d62c6bfc2ef6c7, group_id=7560290c75b64695bdbbbe9694fc3813, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:45.669047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd99679-f4f5-4d27-bbb2-a7099799ba7e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:45.697184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:45.704652 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 199/389] 10.222.0.69 () {58 vars in 1442 bytes} [Fri May 20 08:59:45 2022] PUT /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/groups/7560290c75b64695bdbbbe9694fc3813/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 08:59:45.714517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064674de-6cba-4c9e-b122-f8f42d21743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:45.715400 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064674de-6cba-4c9e-b122-f8f42d21743c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:45.715686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064674de-6cba-4c9e-b122-f8f42d21743c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:45.715962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064674de-6cba-4c9e-b122-f8f42d21743c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:45.728321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064674de-6cba-4c9e-b122-f8f42d21743c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-064674de-6cba-4c9e-b122-f8f42d21743c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:45.737070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:45.738660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:45.739767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 191/390] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:45.744881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd99679-f4f5-4d27-bbb2-a7099799ba7e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:45.867482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd99679-f4f5-4d27-bbb2-a7099799ba7e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:45.910831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:45.911574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:45.911859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:45.912134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:45.912949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:45.913260 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:45.914723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:45.927427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 200/391] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:45.933348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064674de-6cba-4c9e-b122-f8f42d21743c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:45.995880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:46.062183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064674de-6cba-4c9e-b122-f8f42d21743c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:46.097483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eff085-6d6d-4f40-8a3c-6ed9a2a9cea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:46.098391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eff085-6d6d-4f40-8a3c-6ed9a2a9cea4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.098678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eff085-6d6d-4f40-8a3c-6ed9a2a9cea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.098955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eff085-6d6d-4f40-8a3c-6ed9a2a9cea4 None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.100161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eff085-6d6d-4f40-8a3c-6ed9a2a9cea4 None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:46.101640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14eff085-6d6d-4f40-8a3c-6ed9a2a9cea4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:46.103593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 192/392] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:45 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:46.112410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:46.182974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:46.228666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c099b704-2d03-47a6-9e90-cc9795e0ada5 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:46.264788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37490cf8-6aaa-4ed1-ae35-19957b2d66ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:46.265668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490cf8-6aaa-4ed1-ae35-19957b2d66ce None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.265955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490cf8-6aaa-4ed1-ae35-19957b2d66ce None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.266235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37490cf8-6aaa-4ed1-ae35-19957b2d66ce None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.267471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37490cf8-6aaa-4ed1-ae35-19957b2d66ce None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:46.268954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37490cf8-6aaa-4ed1-ae35-19957b2d66ce None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:46.275413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 201/393] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:46 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:46.276956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eff085-6d6d-4f40-8a3c-6ed9a2a9cea4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:46.328642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec2ffb-8579-4409-b727-aa4615c33606 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.329285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec2ffb-8579-4409-b727-aa4615c33606 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.329873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec2ffb-8579-4409-b727-aa4615c33606 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.331885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 202/394] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:46.350074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9ce13-2bc0-45ac-a549-cacca2965fb6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.351136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9ce13-2bc0-45ac-a549-cacca2965fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.352096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab9ce13-2bc0-45ac-a549-cacca2965fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.382742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-bab9ce13-2bc0-45ac-a549-cacca2965fb6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:46.384234 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bab9ce13-2bc0-45ac-a549-cacca2965fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:46.455626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14eff085-6d6d-4f40-8a3c-6ed9a2a9cea4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:46.495126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:46.498004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 203/395] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:46.526917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ec254-5148-41a1-b6d9-f7fe826a40ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:46.528187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ec254-5148-41a1-b6d9-f7fe826a40ce None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.528474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ec254-5148-41a1-b6d9-f7fe826a40ce None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.528748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1ec254-5148-41a1-b6d9-f7fe826a40ce None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/groups/7560290c75b64695bdbbbe9694fc3813/roles/2e550a6a96bb4804926c6213a5af7585` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.533767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ec254-5148-41a1-b6d9-f7fe826a40ce None admin] RBAC: Authorizing `identity:create_grant(project_id=0d512da984d845ada1d62c6bfc2ef6c7, group_id=7560290c75b64695bdbbbe9694fc3813, role_id=2e550a6a96bb4804926c6213a5af7585)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:46.535233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1ec254-5148-41a1-b6d9-f7fe826a40ce None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:46.548729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9ce13-2bc0-45ac-a549-cacca2965fb6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:46.559723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 193/396] 10.222.0.69 () {58 vars in 1442 bytes} [Fri May 20 08:59:46 2022] PUT /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7/groups/7560290c75b64695bdbbbe9694fc3813/roles/2e550a6a96bb4804926c6213a5af7585 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:46.621544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:46.666954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab9ce13-2bc0-45ac-a549-cacca2965fb6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:46.707794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4e24c-3b86-47db-9edc-dd7744f18e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:46.709001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e24c-3b86-47db-9edc-dd7744f18e50 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.709504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e24c-3b86-47db-9edc-dd7744f18e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.709954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4e24c-3b86-47db-9edc-dd7744f18e50 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.721799 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4e24c-3b86-47db-9edc-dd7744f18e50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-18a4e24c-3b86-47db-9edc-dd7744f18e50 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:46.730215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:46.731120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:46.732053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:46.739359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 204/397] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:46.742413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ec254-5148-41a1-b6d9-f7fe826a40ce None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:46.854360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1ec254-5148-41a1-b6d9-f7fe826a40ce None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:46.877373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e707-710e-43d3-81e9-e8701e3048aa None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.877906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e707-710e-43d3-81e9-e8701e3048aa None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.878167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e707-710e-43d3-81e9-e8701e3048aa None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.881042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 205/398] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:46.885713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63db5a-4846-4d47-babb-ae70a7e8b1e1 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.885996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63db5a-4846-4d47-babb-ae70a7e8b1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.886276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63db5a-4846-4d47-babb-ae70a7e8b1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.891440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:46.892297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:46.892583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:46.893727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:46.894532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] RBAC: Adding query filter params (name=member) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:46.894854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:46.896281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:46.904624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ba63db5a-4846-4d47-babb-ae70a7e8b1e1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:46.905949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ba63db5a-4846-4d47-babb-ae70a7e8b1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:46.906570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 194/399] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:46.913160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:46.963620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:46.995342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 206/400] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:47.015852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870d362-e401-4ecf-b0ae-a385ac3e5f88 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.016150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870d362-e401-4ecf-b0ae-a385ac3e5f88 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.016403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870d362-e401-4ecf-b0ae-a385ac3e5f88 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.017907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 207/401] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:47.019070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870d362-e401-4ecf-b0ae-a385ac3e5f88 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.044443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a1657-c702-4ad6-81c1-eae871241789 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.078225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:47.083080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fea9f4-9fe2-4b16-a590-c92d655c75bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.084544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fea9f4-9fe2-4b16-a590-c92d655c75bf None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.084544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fea9f4-9fe2-4b16-a590-c92d655c75bf None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.084976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fea9f4-9fe2-4b16-a590-c92d655c75bf None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.086426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fea9f4-9fe2-4b16-a590-c92d655c75bf None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:47.088512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fea9f4-9fe2-4b16-a590-c92d655c75bf None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:47.090859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 195/402] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:46 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:47.092247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ee9ec-7e7b-486d-a466-d0bbce73641c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.092630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ee9ec-7e7b-486d-a466-d0bbce73641c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.093018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331ee9ec-7e7b-486d-a466-d0bbce73641c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.112942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-331ee9ec-7e7b-486d-a466-d0bbce73641c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:47.114926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-331ee9ec-7e7b-486d-a466-d0bbce73641c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:47.138498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870d362-e401-4ecf-b0ae-a385ac3e5f88 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.174043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaaaa0a-b407-4b71-9d7d-7c2b88233d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.175085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaaa0a-b407-4b71-9d7d-7c2b88233d2d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.175533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaaa0a-b407-4b71-9d7d-7c2b88233d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.175996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaaa0a-b407-4b71-9d7d-7c2b88233d2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.186828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaaaa0a-b407-4b71-9d7d-7c2b88233d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:47.189555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 196/403] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:47.190507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ee9ec-7e7b-486d-a466-d0bbce73641c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-ecaaaa0a-b407-4b71-9d7d-7c2b88233d2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:47.195729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:47.196640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:47.197434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:47.204549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 208/404] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:47.206039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaaaa0a-b407-4b71-9d7d-7c2b88233d2d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.247786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:47.294566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331ee9ec-7e7b-486d-a466-d0bbce73641c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.310139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaaaa0a-b407-4b71-9d7d-7c2b88233d2d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.320693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:47.331636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca268b-441c-4d17-8f83-acd3b3d102f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.332864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca268b-441c-4d17-8f83-acd3b3d102f5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.333290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca268b-441c-4d17-8f83-acd3b3d102f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.333707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ca268b-441c-4d17-8f83-acd3b3d102f5 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.335159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca268b-441c-4d17-8f83-acd3b3d102f5 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:47.336990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ca268b-441c-4d17-8f83-acd3b3d102f5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:47.339051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 197/405] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:47.344537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca268b-441c-4d17-8f83-acd3b3d102f5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.347808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.348590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.349690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.349690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.350161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:47.350620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:47.352221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:47.367175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 209/406] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:47.368354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.454310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ca268b-441c-4d17-8f83-acd3b3d102f5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.481553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:47.489298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632b812-f556-471d-8f58-91874f596b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.490246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632b812-f556-471d-8f58-91874f596b68 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.490630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632b812-f556-471d-8f58-91874f596b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.491102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632b812-f556-471d-8f58-91874f596b68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.497718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfc8d7-80a9-4caa-a17d-1077512b2016 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.503789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632b812-f556-471d-8f58-91874f596b68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-0632b812-f556-471d-8f58-91874f596b68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:47.512440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:47.513516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:47.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:47.516681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 198/407] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:47.517598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632b812-f556-471d-8f58-91874f596b68 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.542219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c3b7-ae98-43e9-aa68-1e2f74f5da3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.543663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c3b7-ae98-43e9-aa68-1e2f74f5da3d None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.543918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c3b7-ae98-43e9-aa68-1e2f74f5da3d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.544228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343c3b7-ae98-43e9-aa68-1e2f74f5da3d None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a/groups/7560290c75b64695bdbbbe9694fc3813/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.547974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343c3b7-ae98-43e9-aa68-1e2f74f5da3d None admin] RBAC: Authorizing `identity:create_grant(project_id=efcdcc966ddd493f83f40b196e076f1a, group_id=7560290c75b64695bdbbbe9694fc3813, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:47.549468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343c3b7-ae98-43e9-aa68-1e2f74f5da3d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:47.566526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 210/408] 10.222.0.69 () {58 vars in 1442 bytes} [Fri May 20 08:59:47 2022] PUT /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a/groups/7560290c75b64695bdbbbe9694fc3813/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:47.576571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c3b7-ae98-43e9-aa68-1e2f74f5da3d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.630744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:47.633038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632b812-f556-471d-8f58-91874f596b68 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.673431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1971d93-5a93-4424-975c-b6ba017ab6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.674821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1971d93-5a93-4424-975c-b6ba017ab6bd None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.675348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1971d93-5a93-4424-975c-b6ba017ab6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.675824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1971d93-5a93-4424-975c-b6ba017ab6bd None admin] PATH_INFO: `/v3/groups/146a740b8dc44628b2f9f26c860f4a82` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.677381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1971d93-5a93-4424-975c-b6ba017ab6bd None admin] RBAC: Authorizing `identity:get_group(group_id=146a740b8dc44628b2f9f26c860f4a82)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:47.679184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1971d93-5a93-4424-975c-b6ba017ab6bd None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:47.685817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 199/409] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/groups/146a740b8dc44628b2f9f26c860f4a82 => generated 229 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:47.687701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1971d93-5a93-4424-975c-b6ba017ab6bd None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.721776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:47.763854 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343c3b7-ae98-43e9-aa68-1e2f74f5da3d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.801940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.802738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.803021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.803433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.804207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:47.804576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:47.805999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:47.815826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 211/410] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:47.821299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.833671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1971d93-5a93-4424-975c-b6ba017ab6bd None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.907125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acae4b-374c-459c-ad12-59bfe07cf544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.908417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acae4b-374c-459c-ad12-59bfe07cf544 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.908909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acae4b-374c-459c-ad12-59bfe07cf544 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.909373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acae4b-374c-459c-ad12-59bfe07cf544 None admin] PATH_INFO: `/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.910813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acae4b-374c-459c-ad12-59bfe07cf544 None admin] RBAC: Authorizing `identity:get_project(project_id=aac440dc4ce14fa1a6d365f33d6d214c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:47.912484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acae4b-374c-459c-ad12-59bfe07cf544 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:47.919376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 200/411] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c => generated 344 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:47.921119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acae4b-374c-459c-ad12-59bfe07cf544 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:47.927664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:47.936477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58ceaf8-6a4e-4bff-a4d3-c4323f8d94b3 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:47.976321 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf38a82-7cff-4100-bb61-2aa96ee20590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:47.977668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf38a82-7cff-4100-bb61-2aa96ee20590 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:47.977668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf38a82-7cff-4100-bb61-2aa96ee20590 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:47.977855 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf38a82-7cff-4100-bb61-2aa96ee20590 None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:47.979090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf38a82-7cff-4100-bb61-2aa96ee20590 None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:47.980651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf38a82-7cff-4100-bb61-2aa96ee20590 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:47.982787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 212/412] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:47.988266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf38a82-7cff-4100-bb61-2aa96ee20590 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:48.028243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:48.031479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acae4b-374c-459c-ad12-59bfe07cf544 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:48.079021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16ec33-7a70-4c92-9136-1e90d30d8829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:48.080346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16ec33-7a70-4c92-9136-1e90d30d8829 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:48.080818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16ec33-7a70-4c92-9136-1e90d30d8829 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:48.081300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16ec33-7a70-4c92-9136-1e90d30d8829 None admin] PATH_INFO: `/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c/groups/146a740b8dc44628b2f9f26c860f4a82/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:48.086547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16ec33-7a70-4c92-9136-1e90d30d8829 None admin] RBAC: Authorizing `identity:create_grant(project_id=aac440dc4ce14fa1a6d365f33d6d214c, group_id=146a740b8dc44628b2f9f26c860f4a82, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:48.088293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16ec33-7a70-4c92-9136-1e90d30d8829 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:48.098696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf38a82-7cff-4100-bb61-2aa96ee20590 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:48.118496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 201/413] 10.222.0.69 () {58 vars in 1442 bytes} [Fri May 20 08:59:47 2022] PUT /identity/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c/groups/146a740b8dc44628b2f9f26c860f4a82/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 08:59:48.136716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525a5e44-1f08-4a44-9aea-e440929c52bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:48.137608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a5e44-1f08-4a44-9aea-e440929c52bf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:48.137897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a5e44-1f08-4a44-9aea-e440929c52bf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:48.138170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525a5e44-1f08-4a44-9aea-e440929c52bf None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:48.139438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a5e44-1f08-4a44-9aea-e440929c52bf None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:48.140910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525a5e44-1f08-4a44-9aea-e440929c52bf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:48.142982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 213/414] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:47 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:48.149251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16ec33-7a70-4c92-9136-1e90d30d8829 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:48.294894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16ec33-7a70-4c92-9136-1e90d30d8829 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:48.339318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3965c-e207-49ed-b624-e0575d291b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:48.340286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3965c-e207-49ed-b624-e0575d291b94 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:48.340572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3965c-e207-49ed-b624-e0575d291b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:48.340844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3965c-e207-49ed-b624-e0575d291b94 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a/groups/7560290c75b64695bdbbbe9694fc3813/roles/2e550a6a96bb4804926c6213a5af7585` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:48.343233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3965c-e207-49ed-b624-e0575d291b94 None admin] RBAC: Authorizing `identity:create_grant(project_id=efcdcc966ddd493f83f40b196e076f1a, group_id=7560290c75b64695bdbbbe9694fc3813, role_id=2e550a6a96bb4804926c6213a5af7585)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:48.344744 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3965c-e207-49ed-b624-e0575d291b94 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:48.375298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 202/415] 10.222.0.69 () {58 vars in 1442 bytes} [Fri May 20 08:59:48 2022] PUT /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a/groups/7560290c75b64695bdbbbe9694fc3813/roles/2e550a6a96bb4804926c6213a5af7585 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 08:59:48.617730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9308d2c-58cc-4622-965d-703923394d92 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:48.618099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9308d2c-58cc-4622-965d-703923394d92 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:48.618210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9308d2c-58cc-4622-965d-703923394d92 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:48.619865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 214/416] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:48.627248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288acc7-abe7-4983-b45a-2cfefd5f52e0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:48.627248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288acc7-abe7-4983-b45a-2cfefd5f52e0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:48.627248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288acc7-abe7-4983-b45a-2cfefd5f52e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:48.644443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-c288acc7-abe7-4983-b45a-2cfefd5f52e0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:48.645931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c288acc7-abe7-4983-b45a-2cfefd5f52e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:48.661297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:48.717430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 203/417] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:48.747925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9308d2c-58cc-4622-965d-703923394d92 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:48.792518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:48.882726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9308d2c-58cc-4622-965d-703923394d92 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:48.917503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aefa54-f973-40d4-89c4-597f41cffa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:48.918308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aefa54-f973-40d4-89c4-597f41cffa82 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:48.918602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aefa54-f973-40d4-89c4-597f41cffa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:48.918863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aefa54-f973-40d4-89c4-597f41cffa82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:48.929635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aefa54-f973-40d4-89c4-597f41cffa82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-58aefa54-f973-40d4-89c4-597f41cffa82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:48.937400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:48.938286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:48.939616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 215/418] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:48.946386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288acc7-abe7-4983-b45a-2cfefd5f52e0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:49.087712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288acc7-abe7-4983-b45a-2cfefd5f52e0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:49.126165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4ae10-a30e-4003-b26d-fc9b1ade6a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:49.127246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4ae10-a30e-4003-b26d-fc9b1ade6a20 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:49.127792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4ae10-a30e-4003-b26d-fc9b1ade6a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:49.128257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4ae10-a30e-4003-b26d-fc9b1ade6a20 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:49.129232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4ae10-a30e-4003-b26d-fc9b1ade6a20 None admin] RBAC: Adding query filter params (name=member) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:49.129726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4ae10-a30e-4003-b26d-fc9b1ade6a20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:49.131331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4ae10-a30e-4003-b26d-fc9b1ade6a20 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:49.140739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 204/419] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:49.148400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aefa54-f973-40d4-89c4-597f41cffa82 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:49.216927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbdb4e-7f22-4d19-b4d0-c068cb9275ce None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:49.217562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbdb4e-7f22-4d19-b4d0-c068cb9275ce None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:49.218034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcbdb4e-7f22-4d19-b4d0-c068cb9275ce None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:49.219972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 205/420] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:49.234987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad837-a6db-46e7-84d6-812c904802a2 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:49.235672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad837-a6db-46e7-84d6-812c904802a2 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:49.236163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ad837-a6db-46e7-84d6-812c904802a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:49.266688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d44ad837-a6db-46e7-84d6-812c904802a2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:49.268258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d44ad837-a6db-46e7-84d6-812c904802a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:49.283311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aefa54-f973-40d4-89c4-597f41cffa82 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:49.285406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:49.307639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:49.315863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30755aa2-a984-4e3d-9041-de6cb70996d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:49.317776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30755aa2-a984-4e3d-9041-de6cb70996d2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:49.318247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30755aa2-a984-4e3d-9041-de6cb70996d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:49.318884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30755aa2-a984-4e3d-9041-de6cb70996d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:49.402790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 206/421] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:49.427758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30755aa2-a984-4e3d-9041-de6cb70996d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:49.429546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30755aa2-a984-4e3d-9041-de6cb70996d2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:49.437228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ad837-a6db-46e7-84d6-812c904802a2 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:49.583543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ad837-a6db-46e7-84d6-812c904802a2 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:49.606571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:49.621692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2e9c2-ac65-41ac-9df6-3e9540befa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:49.622503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e9c2-ac65-41ac-9df6-3e9540befa25 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:49.622798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e9c2-ac65-41ac-9df6-3e9540befa25 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:49.623062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2e9c2-ac65-41ac-9df6-3e9540befa25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:49.627943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 216/422] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:49.635191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30755aa2-a984-4e3d-9041-de6cb70996d2 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:49.645061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2e9c2-ac65-41ac-9df6-3e9540befa25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-4ee2e9c2-ac65-41ac-9df6-3e9540befa25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:49.656071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:49.656963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:49.657756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 207/423] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:49.662998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2e9c2-ac65-41ac-9df6-3e9540befa25 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:49.666155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:49.802759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30755aa2-a984-4e3d-9041-de6cb70996d2 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:49.874978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2e9c2-ac65-41ac-9df6-3e9540befa25 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:49.878453 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab118fb-9385-4aa9-8ee1-f8ddede78545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:49.879407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab118fb-9385-4aa9-8ee1-f8ddede78545 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:49.879690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab118fb-9385-4aa9-8ee1-f8ddede78545 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:49.879961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab118fb-9385-4aa9-8ee1-f8ddede78545 None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:49.882225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab118fb-9385-4aa9-8ee1-f8ddede78545 None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:49.883763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab118fb-9385-4aa9-8ee1-f8ddede78545 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:49.891576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 217/424] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:49 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:49.892690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab118fb-9385-4aa9-8ee1-f8ddede78545 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:49.934974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:49.935586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:49.935865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:49.936134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:49.936896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:49.937257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:49.942811 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:49.966198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 208/425] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:49.971509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:50.001807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:50.080058 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab118fb-9385-4aa9-8ee1-f8ddede78545 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:50.091623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:50.144571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c1c7-5381-46ff-a341-6cfadfa5a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:50.145446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c1c7-5381-46ff-a341-6cfadfa5a75f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.145727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c1c7-5381-46ff-a341-6cfadfa5a75f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.146000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0659c1c7-5381-46ff-a341-6cfadfa5a75f None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.150132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c1c7-5381-46ff-a341-6cfadfa5a75f None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:50.151614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0659c1c7-5381-46ff-a341-6cfadfa5a75f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:50.162981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f22485-5714-4849-9dd1-098d27d9bdba None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:50.164822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 218/426] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:49 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:50.173038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c1c7-5381-46ff-a341-6cfadfa5a75f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:50.238209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568817ae-350b-45cc-bdca-c96acdd5d8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:50.238784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568817ae-350b-45cc-bdca-c96acdd5d8ce None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.239063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568817ae-350b-45cc-bdca-c96acdd5d8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.239355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568817ae-350b-45cc-bdca-c96acdd5d8ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.370511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568817ae-350b-45cc-bdca-c96acdd5d8ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:50.372011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568817ae-350b-45cc-bdca-c96acdd5d8ce None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:50.375491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0659c1c7-5381-46ff-a341-6cfadfa5a75f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:50.431620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:50.447021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:50.456931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbefca24-e735-4f86-bde2-3785c8efd1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:50.457607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefca24-e735-4f86-bde2-3785c8efd1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.457887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefca24-e735-4f86-bde2-3785c8efd1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.458158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbefca24-e735-4f86-bde2-3785c8efd1b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.459016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefca24-e735-4f86-bde2-3785c8efd1b2 None admin] RBAC: Adding query filter params (group.id=7560290c75b64695bdbbbe9694fc3813, role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:50.459339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefca24-e735-4f86-bde2-3785c8efd1b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:50.460865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbefca24-e735-4f86-bde2-3785c8efd1b2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:50.474493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 219/427] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:50 2022] GET /identity/v3/role_assignments?group.id=7560290c75b64695bdbbbe9694fc3813&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 611 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:50.476402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f1a4b-4f1d-45da-8f75-380fb9998c97 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.476921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f1a4b-4f1d-45da-8f75-380fb9998c97 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.477453 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f1a4b-4f1d-45da-8f75-380fb9998c97 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.479312 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 220/428] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:50.485348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7274cc-c148-47a0-85c0-c4418a682b4e None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.485989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7274cc-c148-47a0-85c0-c4418a682b4e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.486495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7274cc-c148-47a0-85c0-c4418a682b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.498683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 209/429] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 20 08:59:50.504279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568817ae-350b-45cc-bdca-c96acdd5d8ce None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:50.508156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4e7274cc-c148-47a0-85c0-c4418a682b4e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:50.509590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4e7274cc-c148-47a0-85c0-c4418a682b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:50.639292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 221/430] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:50.660486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568817ae-350b-45cc-bdca-c96acdd5d8ce None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:50.688780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7274cc-c148-47a0-85c0-c4418a682b4e None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:50.720952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4aa52-03d9-4a77-b681-f159370fae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:50.722018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4aa52-03d9-4a77-b681-f159370fae0a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.722299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4aa52-03d9-4a77-b681-f159370fae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.722568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4aa52-03d9-4a77-b681-f159370fae0a None admin] PATH_INFO: `/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.724007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4aa52-03d9-4a77-b681-f159370fae0a None admin] RBAC: Authorizing `identity:get_project(project_id=0d512da984d845ada1d62c6bfc2ef6c7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:50.725673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4aa52-03d9-4a77-b681-f159370fae0a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:50.728006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 210/431] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:50 2022] GET /identity/v3/projects/0d512da984d845ada1d62c6bfc2ef6c7 => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:50.734058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4aa52-03d9-4a77-b681-f159370fae0a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:50.781711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:50.887688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4aa52-03d9-4a77-b681-f159370fae0a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:50.890375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7274cc-c148-47a0-85c0-c4418a682b4e None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:50.912637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:50.935408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e0547d-98ba-4d9c-b3ce-f4ee1efa7e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:50.936570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0547d-98ba-4d9c-b3ce-f4ee1efa7e36 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.936697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74309b-4ba5-4a7f-9835-f18100277cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:50.936897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0547d-98ba-4d9c-b3ce-f4ee1efa7e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.937136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e0547d-98ba-4d9c-b3ce-f4ee1efa7e36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.937459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74309b-4ba5-4a7f-9835-f18100277cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.937930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74309b-4ba5-4a7f-9835-f18100277cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.938205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74309b-4ba5-4a7f-9835-f18100277cf6 None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.940542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74309b-4ba5-4a7f-9835-f18100277cf6 None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:50.940958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74309b-4ba5-4a7f-9835-f18100277cf6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:50.942933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 211/432] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:50 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:50.944768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ca4d-5b9f-4269-aacf-9588a90a55e9 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.945189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ca4d-5b9f-4269-aacf-9588a90a55e9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.945549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4ca4d-5b9f-4269-aacf-9588a90a55e9 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.947022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 212/433] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:50.949328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4ca4d-5b9f-4269-aacf-9588a90a55e9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:50.960739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e0547d-98ba-4d9c-b3ce-f4ee1efa7e36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-23e0547d-98ba-4d9c-b3ce-f4ee1efa7e36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:50.970212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:50.971173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 20 08:59:50.972017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:50.973113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 222/434] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:50.974098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7306c21-7de9-4cdc-a0a3-2fc619e806b5 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:50.974384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7306c21-7de9-4cdc-a0a3-2fc619e806b5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:50.974687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7306c21-7de9-4cdc-a0a3-2fc619e806b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:50.996491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c7306c21-7de9-4cdc-a0a3-2fc619e806b5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:50.997665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c7306c21-7de9-4cdc-a0a3-2fc619e806b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:51.077452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 223/435] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:51.078252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7306c21-7de9-4cdc-a0a3-2fc619e806b5 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:51.092950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:51.101732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa4ca4d-5b9f-4269-aacf-9588a90a55e9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:51.147148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:51.147908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.148220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.148498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.149346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] RBAC: Adding query filter params (group.id=7560290c75b64695bdbbbe9694fc3813, role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:51.149654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:51.151214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:51.157840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 213/436] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:50 2022] GET /identity/v3/role_assignments?group.id=7560290c75b64695bdbbbe9694fc3813&scope.project.id=0d512da984d845ada1d62c6bfc2ef6c7&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:51.158743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:51.209859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7306c21-7de9-4cdc-a0a3-2fc619e806b5 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:51.222774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:51.250430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48e2ed-f2ac-47bf-9a27-2fbc87d83680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:51.251122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48e2ed-f2ac-47bf-9a27-2fbc87d83680 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.251438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48e2ed-f2ac-47bf-9a27-2fbc87d83680 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.251717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48e2ed-f2ac-47bf-9a27-2fbc87d83680 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.252499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48e2ed-f2ac-47bf-9a27-2fbc87d83680 None admin] RBAC: Adding query filter params (name=member) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:51.252807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48e2ed-f2ac-47bf-9a27-2fbc87d83680 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:51.254352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48e2ed-f2ac-47bf-9a27-2fbc87d83680 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:51.269008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 224/437] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:51.270698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e2edb-cafc-4b40-8291-f2973f308a4a None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.271064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e2edb-cafc-4b40-8291-f2973f308a4a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.271423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762e2edb-cafc-4b40-8291-f2973f308a4a None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.272927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 225/438] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:51.275707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e2edb-cafc-4b40-8291-f2973f308a4a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:51.283673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b703d2b-ba68-44ad-8f62-4433a33ac62b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:51.326489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4878f5fb-3714-4a30-b2f9-61c1b83b1c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:51.327336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f5fb-3714-4a30-b2f9-61c1b83b1c14 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.327787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f5fb-3714-4a30-b2f9-61c1b83b1c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.327900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878f5fb-3714-4a30-b2f9-61c1b83b1c14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.340152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4878f5fb-3714-4a30-b2f9-61c1b83b1c14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-4878f5fb-3714-4a30-b2f9-61c1b83b1c14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:51.347809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:51.348731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:51.349819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 214/439] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:51.351656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53630327-73ac-4a97-b060-b7c972afc9ee None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.351656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53630327-73ac-4a97-b060-b7c972afc9ee None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.351656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53630327-73ac-4a97-b060-b7c972afc9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.370613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-53630327-73ac-4a97-b060-b7c972afc9ee None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:51.372317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-53630327-73ac-4a97-b060-b7c972afc9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:51.394301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:51.397722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762e2edb-cafc-4b40-8291-f2973f308a4a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:51.439197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823f4a-1f3a-40f0-b63f-ba2db5e15cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:51.439876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823f4a-1f3a-40f0-b63f-ba2db5e15cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.441344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823f4a-1f3a-40f0-b63f-ba2db5e15cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.441344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b823f4a-1f3a-40f0-b63f-ba2db5e15cb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.445778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 215/440] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:51.446906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53630327-73ac-4a97-b060-b7c972afc9ee None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:51.487797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:51.522429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823f4a-1f3a-40f0-b63f-ba2db5e15cb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:51.523820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b823f4a-1f3a-40f0-b63f-ba2db5e15cb7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:51.579843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53630327-73ac-4a97-b060-b7c972afc9ee None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:51.616291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:51.616984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.617260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.617530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.618279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:51.618580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:51.619996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:51.628382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 216/441] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:51.629449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:51.639147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 226/442] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:51.640062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823f4a-1f3a-40f0-b63f-ba2db5e15cb7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:51.650870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:51.709061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:51.730184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b086a2-8dbe-427c-9b58-0661316afa27 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:51.738204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b823f4a-1f3a-40f0-b63f-ba2db5e15cb7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:51.762261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a277c-4d17-44cf-8b4d-6795f6a281f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:51.763064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a277c-4d17-44cf-8b4d-6795f6a281f6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.763372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a277c-4d17-44cf-8b4d-6795f6a281f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.763647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43a277c-4d17-44cf-8b4d-6795f6a281f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.773771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef408-d7cc-4c44-9b53-1d0a7fc15bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:51.774424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef408-d7cc-4c44-9b53-1d0a7fc15bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.774721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef408-d7cc-4c44-9b53-1d0a7fc15bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.775013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ef408-d7cc-4c44-9b53-1d0a7fc15bd1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.779840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43a277c-4d17-44cf-8b4d-6795f6a281f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-a43a277c-4d17-44cf-8b4d-6795f6a281f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:51.787915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:51.788954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:51.790035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 217/443] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:51.791380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a277c-4d17-44cf-8b4d-6795f6a281f6 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:51.849280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ef408-d7cc-4c44-9b53-1d0a7fc15bd1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:51.850704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9ef408-d7cc-4c44-9b53-1d0a7fc15bd1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:51.866532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:51.889267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43a277c-4d17-44cf-8b4d-6795f6a281f6 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:51.919930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:51.926290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce6e6a-17a4-45f1-9ade-a9f2447598f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:51.927167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce6e6a-17a4-45f1-9ade-a9f2447598f7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:51.927468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce6e6a-17a4-45f1-9ade-a9f2447598f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:51.927743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ce6e6a-17a4-45f1-9ade-a9f2447598f7 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:51.928942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce6e6a-17a4-45f1-9ade-a9f2447598f7 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:51.930458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ce6e6a-17a4-45f1-9ade-a9f2447598f7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:51.932193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 218/444] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:51.932982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce6e6a-17a4-45f1-9ade-a9f2447598f7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:51.954842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 227/445] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:51.955715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef408-d7cc-4c44-9b53-1d0a7fc15bd1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.033619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ce6e6a-17a4-45f1-9ade-a9f2447598f7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.056079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ef408-d7cc-4c44-9b53-1d0a7fc15bd1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.067025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.067716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.067996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.068269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.069015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:52.069316 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.070723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.078441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 219/446] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:52.079205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.090577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333869c-bb59-4972-86bc-76828751840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.091113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:52.091475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333869c-bb59-4972-86bc-76828751840e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.091758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333869c-bb59-4972-86bc-76828751840e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.092051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b333869c-bb59-4972-86bc-76828751840e None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.093266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333869c-bb59-4972-86bc-76828751840e None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.094875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b333869c-bb59-4972-86bc-76828751840e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.096544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 228/447] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:51 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:52.097295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333869c-bb59-4972-86bc-76828751840e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.139845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:52.182745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895e008-47f7-4d54-9cfa-249d12d64439 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.195068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b333869c-bb59-4972-86bc-76828751840e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.214335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcacbd0-affc-4f88-9e4c-76c014a1b580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.215219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcacbd0-affc-4f88-9e4c-76c014a1b580 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.215529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcacbd0-affc-4f88-9e4c-76c014a1b580 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.215804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcacbd0-affc-4f88-9e4c-76c014a1b580 None admin] PATH_INFO: `/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.216969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcacbd0-affc-4f88-9e4c-76c014a1b580 None admin] RBAC: Authorizing `identity:get_project(project_id=aac440dc4ce14fa1a6d365f33d6d214c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.218435 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcacbd0-affc-4f88-9e4c-76c014a1b580 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.220193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 220/448] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:52 2022] GET /identity/v3/projects/aac440dc4ce14fa1a6d365f33d6d214c => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:52.220946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcacbd0-affc-4f88-9e4c-76c014a1b580 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.232719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8478b7d-afb7-4ed7-b521-53c4ab1f68ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.233378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478b7d-afb7-4ed7-b521-53c4ab1f68ac None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.233655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478b7d-afb7-4ed7-b521-53c4ab1f68ac None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.233926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8478b7d-afb7-4ed7-b521-53c4ab1f68ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.299275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:52.301947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8478b7d-afb7-4ed7-b521-53c4ab1f68ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.303448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8478b7d-afb7-4ed7-b521-53c4ab1f68ac None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.323705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcacbd0-affc-4f88-9e4c-76c014a1b580 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.343034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:52.355639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.356283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.356560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.356839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.357661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] RBAC: Adding query filter params (group.id=7560290c75b64695bdbbbe9694fc3813, role.id=580338b3f2b847c78aef559dbaaf51e4, scope.project.id=efcdcc966ddd493f83f40b196e076f1a) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:52.357968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.359422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.365043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 221/449] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:52 2022] GET /identity/v3/role_assignments?group.id=7560290c75b64695bdbbbe9694fc3813&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=580338b3f2b847c78aef559dbaaf51e4 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:52.365801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.421079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 229/450] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 08:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 08:59:52.427398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8478b7d-afb7-4ed7-b521-53c4ab1f68ac None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.495360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a48d2-3101-491f-8f9a-14e73d989749 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.533662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9192a-e8f3-4fc2-951a-050a99b3f4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.534870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9192a-e8f3-4fc2-951a-050a99b3f4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.535153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9192a-e8f3-4fc2-951a-050a99b3f4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.535459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9192a-e8f3-4fc2-951a-050a99b3f4b4 None admin] PATH_INFO: `/v3/groups/146a740b8dc44628b2f9f26c860f4a82` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.536681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9192a-e8f3-4fc2-951a-050a99b3f4b4 None admin] RBAC: Authorizing `identity:get_group(group_id=146a740b8dc44628b2f9f26c860f4a82)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.538162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9192a-e8f3-4fc2-951a-050a99b3f4b4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.539911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 222/451] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:52 2022] GET /identity/v3/groups/146a740b8dc44628b2f9f26c860f4a82 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:52.543548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8478b7d-afb7-4ed7-b521-53c4ab1f68ac None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.546006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9192a-e8f3-4fc2-951a-050a99b3f4b4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.555587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:52.585632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484500d-eaff-4511-aa67-cd4aa2853f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.586766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484500d-eaff-4511-aa67-cd4aa2853f77 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.587044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484500d-eaff-4511-aa67-cd4aa2853f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.587399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484500d-eaff-4511-aa67-cd4aa2853f77 None admin] PATH_INFO: `/v3/projects/efcdcc966ddd493f83f40b196e076f1a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.588593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484500d-eaff-4511-aa67-cd4aa2853f77 None admin] RBAC: Authorizing `identity:get_project(project_id=efcdcc966ddd493f83f40b196e076f1a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.590053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a484500d-eaff-4511-aa67-cd4aa2853f77 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.596484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 230/452] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 08:59:52 2022] GET /identity/v3/projects/efcdcc966ddd493f83f40b196e076f1a => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:52.597317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484500d-eaff-4511-aa67-cd4aa2853f77 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.608627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:52.668879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9192a-e8f3-4fc2-951a-050a99b3f4b4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.708131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.708713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.708985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.709256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.710101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] RBAC: Adding query filter params (group.id=146a740b8dc44628b2f9f26c860f4a82, role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=aac440dc4ce14fa1a6d365f33d6d214c) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:52.710412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.711978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.718190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484500d-eaff-4511-aa67-cd4aa2853f77 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.722197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 223/453] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:52 2022] GET /identity/v3/role_assignments?group.id=146a740b8dc44628b2f9f26c860f4a82&scope.project.id=aac440dc4ce14fa1a6d365f33d6d214c&role.id=6dca675ade814980a252f800fb4fa3ad => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:52.759579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2a4c6-0646-48a4-a2c9-3d6e0e453aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.760485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a4c6-0646-48a4-a2c9-3d6e0e453aef None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.760763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a4c6-0646-48a4-a2c9-3d6e0e453aef None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.761025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2a4c6-0646-48a4-a2c9-3d6e0e453aef None admin] PATH_INFO: `/v3/groups/7560290c75b64695bdbbbe9694fc3813` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.763666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2a4c6-0646-48a4-a2c9-3d6e0e453aef None admin] RBAC: Authorizing `identity:get_group(group_id=7560290c75b64695bdbbbe9694fc3813)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.765177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2a4c6-0646-48a4-a2c9-3d6e0e453aef None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.767176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 231/454] 10.222.0.69 () {56 vars in 1180 bytes} [Fri May 20 08:59:52 2022] GET /identity/v3/groups/7560290c75b64695bdbbbe9694fc3813 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 08:59:52.775200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:52.849105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:52.883231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280a2ea-5cd3-46a0-8562-c48f56034669 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:52.918552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08225ae0-f483-456b-931f-154747c35798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:52.919204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08225ae0-f483-456b-931f-154747c35798 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:52.919543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08225ae0-f483-456b-931f-154747c35798 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:52.919793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08225ae0-f483-456b-931f-154747c35798 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:52.920630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08225ae0-f483-456b-931f-154747c35798 None admin] RBAC: Adding query filter params (group.id=7560290c75b64695bdbbbe9694fc3813, role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=efcdcc966ddd493f83f40b196e076f1a) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:52.920938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08225ae0-f483-456b-931f-154747c35798 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:52.922351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08225ae0-f483-456b-931f-154747c35798 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:52.928275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 224/455] 10.222.0.69 () {56 vars in 1376 bytes} [Fri May 20 08:59:52 2022] GET /identity/v3/role_assignments?group.id=7560290c75b64695bdbbbe9694fc3813&scope.project.id=efcdcc966ddd493f83f40b196e076f1a&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:55.638132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b1523-43a6-4721-a3f5-d528d148247d None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:55.638692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b1523-43a6-4721-a3f5-d528d148247d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:55.638692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b1523-43a6-4721-a3f5-d528d148247d None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:55.640359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 232/456] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:55.645278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb85b-0dfd-40e8-80ba-65939e951456 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:55.645706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb85b-0dfd-40e8-80ba-65939e951456 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:55.645869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42cb85b-0dfd-40e8-80ba-65939e951456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:55.666294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-c42cb85b-0dfd-40e8-80ba-65939e951456 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:55.667503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c42cb85b-0dfd-40e8-80ba-65939e951456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:55.756573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 225/457] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:55.801094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b1523-43a6-4721-a3f5-d528d148247d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:55.911537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:55.944928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b1523-43a6-4721-a3f5-d528d148247d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:55.965357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d13109-c1c6-4d78-bc61-0a0866e94d09 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:55.966189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d13109-c1c6-4d78-bc61-0a0866e94d09 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:55.967031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d13109-c1c6-4d78-bc61-0a0866e94d09 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:55.969001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 226/458] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:55.974344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a941890-89e0-4f69-b55f-dc47c863bb5b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:55.974777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a941890-89e0-4f69-b55f-dc47c863bb5b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:55.975280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a941890-89e0-4f69-b55f-dc47c863bb5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:55.986271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:55.987102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:55.987438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:55.987709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.004981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:56.006497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:56.019792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5a941890-89e0-4f69-b55f-dc47c863bb5b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:56.021006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5a941890-89e0-4f69-b55f-dc47c863bb5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 08:59:56.038833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:56.039786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:56.041498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 08:59:56.041498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 08:59:56.041498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:56.041498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:56.041498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 233/459] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:56.046724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:56.123673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:56.128901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 227/460] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:56.155745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a941890-89e0-4f69-b55f-dc47c863bb5b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:56.247921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645e7ac1-0d6e-445e-8c42-e4586caf6472 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:56.281667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a941890-89e0-4f69-b55f-dc47c863bb5b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:56.285548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:56.295253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d841d72-99e1-4708-ba6e-eed28c0843dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:56.296476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841d72-99e1-4708-ba6e-eed28c0843dc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.296851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841d72-99e1-4708-ba6e-eed28c0843dc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.297361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d841d72-99e1-4708-ba6e-eed28c0843dc None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.298416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d841d72-99e1-4708-ba6e-eed28c0843dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:56.298723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d841d72-99e1-4708-ba6e-eed28c0843dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:56.300306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d841d72-99e1-4708-ba6e-eed28c0843dc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:56.319003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 234/461] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:56.320579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1859d2c-2ce4-4441-a096-283cff440b41 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.320872 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1859d2c-2ce4-4441-a096-283cff440b41 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.321153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1859d2c-2ce4-4441-a096-283cff440b41 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.322518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 235/462] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:56.323618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d75eb-076e-45e8-9288-6c1942f9534a None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.323907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d75eb-076e-45e8-9288-6c1942f9534a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.324187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d75eb-076e-45e8-9288-6c1942f9534a None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.329029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7cb87-3980-406e-bd4a-660c19563cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:56.329493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 236/463] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:56.330034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cb87-3980-406e-bd4a-660c19563cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.330279 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d75eb-076e-45e8-9288-6c1942f9534a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:56.330373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cb87-3980-406e-bd4a-660c19563cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.330565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7cb87-3980-406e-bd4a-660c19563cb2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.342897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7cb87-3980-406e-bd4a-660c19563cb2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:56.344712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7cb87-3980-406e-bd4a-660c19563cb2 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-d7a7cb87-3980-406e-bd4a-660c19563cb2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 08:59:56.361865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:56.362733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:56.364023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 08:59:56.364023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 08:59:56.364023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:56.364023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:56.364023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 228/464] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 08:59:56.364634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247da231-d4a2-46be-82be-1ea8ebac2e69 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.365050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247da231-d4a2-46be-82be-1ea8ebac2e69 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.365156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247da231-d4a2-46be-82be-1ea8ebac2e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.381762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-247da231-d4a2-46be-82be-1ea8ebac2e69 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:56.382986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-247da231-d4a2-46be-82be-1ea8ebac2e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:56.434113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:56.448767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d75eb-076e-45e8-9288-6c1942f9534a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:56.460987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 229/465] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:56.462492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40dbca-4236-459d-97aa-f3721445e83e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.462941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40dbca-4236-459d-97aa-f3721445e83e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.463459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a40dbca-4236-459d-97aa-f3721445e83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.479893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8a40dbca-4236-459d-97aa-f3721445e83e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:56.481074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8a40dbca-4236-459d-97aa-f3721445e83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:56.505545 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c732e-7a89-4af4-b823-890333fb86c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:56.506228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c732e-7a89-4af4-b823-890333fb86c0 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.506511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c732e-7a89-4af4-b823-890333fb86c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.506788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93c732e-7a89-4af4-b823-890333fb86c0 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.507553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c732e-7a89-4af4-b823-890333fb86c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:56.509200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93c732e-7a89-4af4-b823-890333fb86c0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:56.534921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 237/466] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:56 2022] POST /identity/v3/users => generated 246 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:56.536087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c732e-7a89-4af4-b823-890333fb86c0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:56.563910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 230/467] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:56.571016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40dbca-4236-459d-97aa-f3721445e83e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:56.612275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:56.689151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93c732e-7a89-4af4-b823-890333fb86c0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:56.704920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a40dbca-4236-459d-97aa-f3721445e83e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:56.713200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:56.736759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:56.737240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.737527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.737816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.739680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:56.739680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:56.741110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:56.743786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:56.744586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.744862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.745132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.754835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 238/468] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:56.756174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:56.758562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:56.760494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 08:59:56.783709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:56.784617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:56.786905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 08:59:56.786905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 08:59:56.786905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:56.786905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:56.786905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 231/469] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:56.786905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:56.889976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef325cde-bdfc-44eb-bee7-6d3bbf3af8bc None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:56.892979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:56.902131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d0f15-10a9-400e-999f-9a7f81edef16 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:56.927926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:56.929187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.929803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.930266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.937870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6025144-1494-4ec8-854b-db4026cf2977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:56.938252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025144-1494-4ec8-854b-db4026cf2977 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:56.938465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025144-1494-4ec8-854b-db4026cf2977 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:56.938741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6025144-1494-4ec8-854b-db4026cf2977 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:56.939475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6025144-1494-4ec8-854b-db4026cf2977 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:56.940921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6025144-1494-4ec8-854b-db4026cf2977 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:56.942899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:56.944611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:56.968678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 232/470] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:56 2022] POST /identity/v3/users => generated 248 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 08:59:56.968678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6025144-1494-4ec8-854b-db4026cf2977 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 08:59:56.968932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:56.970334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:56.971192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 08:59:56.971192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 08:59:56.971192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:56.971192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:56.971192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 239/471] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:56.975032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:57.006305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:57.172522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6025144-1494-4ec8-854b-db4026cf2977 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:57.204011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b9b7d-58ad-4fe5-ab11-20637d71f0c7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:57.221976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:57.222835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:57.223251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:57.223558 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:57.224321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:57.224812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:57.227068 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:57.243150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 233/472] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:57.248926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:57.252858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:57.253537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:57.253822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:57.254096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:57.254835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:57.255144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:57.256661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:57.274421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 240/473] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 08:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:57.275773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:57.295736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:57.337869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:57.366477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5101f2fa-b52b-4265-ba9e-2a8372b715a9 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:57.380915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdab12-cfa9-408a-89fc-856ac91a784f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:57.406996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37cb85f-2f1b-4d61-85cb-3a4bd0c7c1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:57.408064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cb85f-2f1b-4d61-85cb-3a4bd0c7c1ea None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:57.408650 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cb85f-2f1b-4d61-85cb-3a4bd0c7c1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:57.409113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37cb85f-2f1b-4d61-85cb-3a4bd0c7c1ea None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:57.410051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37cb85f-2f1b-4d61-85cb-3a4bd0c7c1ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:57.411806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37cb85f-2f1b-4d61-85cb-3a4bd0c7c1ea None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:57.417195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828fe41-f6a4-44c6-bb31-93ca833a4dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:57.417879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828fe41-f6a4-44c6-bb31-93ca833a4dc1 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:57.418162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828fe41-f6a4-44c6-bb31-93ca833a4dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:57.418439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828fe41-f6a4-44c6-bb31-93ca833a4dc1 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:57.419162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828fe41-f6a4-44c6-bb31-93ca833a4dc1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:57.420651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9828fe41-f6a4-44c6-bb31-93ca833a4dc1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:57.438553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 234/474] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:57 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 08:59:57.440184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502d1f-b4dd-45ec-b734-cd5031d71f98 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:57.440613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502d1f-b4dd-45ec-b734-cd5031d71f98 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:57.441021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd502d1f-b4dd-45ec-b734-cd5031d71f98 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:57.442971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 235/475] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:57.449203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e8afd-8527-4cdd-a7ed-ce47bdf56163 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:57.449854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e8afd-8527-4cdd-a7ed-ce47bdf56163 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:57.450317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e8afd-8527-4cdd-a7ed-ce47bdf56163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:57.450722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 241/476] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 08:59:57 2022] POST /identity/v3/users => generated 249 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 08:59:57.476863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-569e8afd-8527-4cdd-a7ed-ce47bdf56163 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:57.478236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-569e8afd-8527-4cdd-a7ed-ce47bdf56163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:57.598491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 236/477] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 08:59:57.633426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828fe41-f6a4-44c6-bb31-93ca833a4dc1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:57.722512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:57.762210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9828fe41-f6a4-44c6-bb31-93ca833a4dc1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:57.802024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:57.802709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:57.803037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:57.803353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:57.804167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:57.805638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:57.814193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.sql.core [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 20 08:59:57.814193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 20 08:59:57.814193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [parameters: {'id': '497617c620664c43a9a93043a20292d2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 20 08:59:57.814193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80637) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.dialect.do_execute( May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result = self._query(query) May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application conn.query(q) May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result.read() May 20 08:59:57.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application packet.raise_for_error() May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return ref.to_dict() May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 20 08:59:57.832195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application next(self.gen) May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application yield resource May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application next(self.gen) May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.session.rollback() May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.force_reraise() May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise self.value May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application session.commit() May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 20 08:59:57.833091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._prepare_impl() May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.session.flush() May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._flush(objects) May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application compat.raise_( May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application flush_context.execute() May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rec.execute(self) May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 20 08:59:57.834547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application _emit_insert_statements( May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application c = connection._execute_20( May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ret = self._execute_context( May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 20 08:59:57.836792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.dialect.do_execute( May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result = self._query(query) May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application conn.query(q) May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result.read() May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 20 08:59:57.838736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application packet.raise_for_error() May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [parameters: {'id': '497617c620664c43a9a93043a20292d2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:57.840626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 20 08:59:57.846043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 08:59:57.847136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 242/478] 10.222.0.69 () {60 vars in 1131 bytes} [Fri May 20 08:59:57 2022] POST /identity/v3/roles => generated 144 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:57.847216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e8afd-8527-4cdd-a7ed-ce47bdf56163 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:57.914111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:57.974996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e8afd-8527-4cdd-a7ed-ce47bdf56163 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:58.024399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9a730-6e60-4d7c-b86a-54151944712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:58.025209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9a730-6e60-4d7c-b86a-54151944712a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:58.025486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9a730-6e60-4d7c-b86a-54151944712a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:58.025756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9a730-6e60-4d7c-b86a-54151944712a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:58.036351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9a730-6e60-4d7c-b86a-54151944712a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-70e9a730-6e60-4d7c-b86a-54151944712a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:58.044820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:58.045708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:58.046531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 237/479] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 08:59:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:58.051734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:58.192871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28fafdd-e270-49ea-adbd-925c0abf8d91 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:58.227507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22a32-e36d-4efb-bcb2-a6a7529f4361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:58.228184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22a32-e36d-4efb-bcb2-a6a7529f4361 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:58.228459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22a32-e36d-4efb-bcb2-a6a7529f4361 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:58.228727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22a32-e36d-4efb-bcb2-a6a7529f4361 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:58.229480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22a32-e36d-4efb-bcb2-a6a7529f4361 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:58.229787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22a32-e36d-4efb-bcb2-a6a7529f4361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:58.231163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22a32-e36d-4efb-bcb2-a6a7529f4361 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:58.233465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:58.240508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 243/480] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 08:59:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:59.190488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504de08-09c8-4e83-95bb-6e44dedbed78 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:59.190488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504de08-09c8-4e83-95bb-6e44dedbed78 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:59.190488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504de08-09c8-4e83-95bb-6e44dedbed78 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:59.191489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 238/481] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 08:59:59.197057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c4044-ec83-41e1-9818-3fb293977174 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:59.197873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c4044-ec83-41e1-9818-3fb293977174 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:59.198361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c4044-ec83-41e1-9818-3fb293977174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:59.216022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-eb2c4044-ec83-41e1-9818-3fb293977174 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:59.217574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-eb2c4044-ec83-41e1-9818-3fb293977174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 08:59:59.304178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 244/482] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 08:59:59.340766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504de08-09c8-4e83-95bb-6e44dedbed78 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:59.468871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504de08-09c8-4e83-95bb-6e44dedbed78 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:59.492596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:59.512454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ebec0-4f97-4f05-a0f6-2ead789b1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:59.516333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ebec0-4f97-4f05-a0f6-2ead789b1e7b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:59.516828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ebec0-4f97-4f05-a0f6-2ead789b1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:59.517327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10ebec0-4f97-4f05-a0f6-2ead789b1e7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:59.534287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10ebec0-4f97-4f05-a0f6-2ead789b1e7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b10ebec0-4f97-4f05-a0f6-2ead789b1e7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 08:59:59.542769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:59.543700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 08:59:59.544488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:59.546848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 239/483] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 08:59:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:59.552829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c4044-ec83-41e1-9818-3fb293977174 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:59.677071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c4044-ec83-41e1-9818-3fb293977174 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:59.712887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 08:59:59.721230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:59.721919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:59.722210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:59.722473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:59.723292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 08:59:59.723892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:59.725368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:59.735559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 245/484] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 08:59:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 08:59:59.744507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ebec0-4f97-4f05-a0f6-2ead789b1e7b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:59.880889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10ebec0-4f97-4f05-a0f6-2ead789b1e7b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 08:59:59.913644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d94bd-fc50-4305-85f6-5198c521046b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 08:59:59.914445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d94bd-fc50-4305-85f6-5198c521046b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:59.914719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d94bd-fc50-4305-85f6-5198c521046b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:59.914985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d94bd-fc50-4305-85f6-5198c521046b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:59.926043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d94bd-fc50-4305-85f6-5198c521046b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 08:59:59.927545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d94bd-fc50-4305-85f6-5198c521046b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-d58d94bd-fc50-4305-85f6-5198c521046b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 08:59:59.944262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 08:59:59.945161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 08:59:59.946421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 08:59:59.946421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 08:59:59.946421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 08:59:59.946421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 08:59:59.946421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 240/485] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 08:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 08:59:59.952510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 08:59:59.959239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20992ea-4ae4-4d5e-98da-576f001d6c38 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:59.959546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20992ea-4ae4-4d5e-98da-576f001d6c38 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:59.959821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20992ea-4ae4-4d5e-98da-576f001d6c38 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:59.961315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 241/486] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 08:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 08:59:59.966297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fcae-588c-46ed-97ea-5790d7f23634 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 08:59:59.966889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fcae-588c-46ed-97ea-5790d7f23634 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 08:59:59.967402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587fcae-588c-46ed-97ea-5790d7f23634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 08:59:59.994673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8587fcae-588c-46ed-97ea-5790d7f23634 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 08:59:59.997182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8587fcae-588c-46ed-97ea-5790d7f23634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:00.049894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:00.131599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 242/487] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 08:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 09:00:00.143110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39500eb5-ce26-4cbe-bd18-1793c4b0c6a4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:00.150236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587fcae-588c-46ed-97ea-5790d7f23634 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:00.209483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:00.210917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.211448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.211943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.212874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:00.213370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:00.216171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:00.234265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 246/488] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 08:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:00.245169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:00.289176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587fcae-588c-46ed-97ea-5790d7f23634 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:00.326142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c873a-8cf7-4632-8901-15c51f51bd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:00.327057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c873a-8cf7-4632-8901-15c51f51bd9e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.327496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c873a-8cf7-4632-8901-15c51f51bd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.327839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c873a-8cf7-4632-8901-15c51f51bd9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.334246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:00.339999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c873a-8cf7-4632-8901-15c51f51bd9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-6c4c873a-8cf7-4632-8901-15c51f51bd9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:00.348013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:00.349126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:00.350135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:00.352623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 243/489] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:00.354069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b23d-8c1c-46a0-b987-cead85d8daca None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.354493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b23d-8c1c-46a0-b987-cead85d8daca None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.354906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a057b23d-8c1c-46a0-b987-cead85d8daca None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.356585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 244/490] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:00.358034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057b23d-8c1c-46a0-b987-cead85d8daca None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:00.393269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:00.396270 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4fe6b9-21f8-4b96-897f-fd57f877b551 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:00.433741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50d04e9-5dde-4da2-94e2-ef795d3f8afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:00.434634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d04e9-5dde-4da2-94e2-ef795d3f8afe None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.434978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d04e9-5dde-4da2-94e2-ef795d3f8afe None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.435398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50d04e9-5dde-4da2-94e2-ef795d3f8afe None admin] PATH_INFO: `/v3/users/nova` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.443698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d04e9-5dde-4da2-94e2-ef795d3f8afe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:00.445385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50d04e9-5dde-4da2-94e2-ef795d3f8afe None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-a50d04e9-5dde-4da2-94e2-ef795d3f8afe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:00.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:00.463160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:00.464106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:00.468209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 247/491] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:00:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:00.469559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919d27f-28d1-4fcb-8465-6d0e49734815 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.469850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919d27f-28d1-4fcb-8465-6d0e49734815 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.470128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919d27f-28d1-4fcb-8465-6d0e49734815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.481487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a057b23d-8c1c-46a0-b987-cead85d8daca None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:00.489130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f919d27f-28d1-4fcb-8465-6d0e49734815 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:00.490335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f919d27f-28d1-4fcb-8465-6d0e49734815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:00.522255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:00.522979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.523289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.523565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.524321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:00.524625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:00.526015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:00.534814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 245/492] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:00.539604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:00.571399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 248/493] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:00.572417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919d27f-28d1-4fcb-8465-6d0e49734815 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:00.627104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:00.643674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:00.647947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c9ff2b-c415-4e2e-a588-23f6056c6332 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:00.682505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:00.683176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.683482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.683756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.684630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:00.684900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:00.686334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:00.707146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919d27f-28d1-4fcb-8465-6d0e49734815 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:00.723897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 246/494] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:00:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:00.724870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:00.748289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:00.749218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.749557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.749826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.761473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:00.763032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:00.785006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:00.785941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:00.786768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:00.786768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:00.786768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:00.786768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:00.790825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 249/495] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:00.791753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:00.829439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38409086-943c-40bc-9062-0e6f01317c41 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:00.865563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4976f-dad3-45b8-abbe-5133fc1f3ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:00.866357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4976f-dad3-45b8-abbe-5133fc1f3ffd None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.866631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4976f-dad3-45b8-abbe-5133fc1f3ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.866944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4976f-dad3-45b8-abbe-5133fc1f3ffd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.877987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4976f-dad3-45b8-abbe-5133fc1f3ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-65b4976f-dad3-45b8-abbe-5133fc1f3ffd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:00.885585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:00.886495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:00.887411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 247/496] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:00.888385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc295e-73bb-449a-bda8-603faecb9a7b None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.888385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc295e-73bb-449a-bda8-603faecb9a7b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.888651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbc295e-73bb-449a-bda8-603faecb9a7b None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:00.890169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 248/497] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:00.890912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc295e-73bb-449a-bda8-603faecb9a7b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:00.898343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:00.901235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:00.911673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365a04e-5c2c-4d96-86c6-da95390e6139 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:00.947415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd4cee-1201-4914-a181-d4da9349da98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:00.948147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4cee-1201-4914-a181-d4da9349da98 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:00.948434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4cee-1201-4914-a181-d4da9349da98 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:00.948711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bd4cee-1201-4914-a181-d4da9349da98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.003946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbc295e-73bb-449a-bda8-603faecb9a7b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.015841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd4cee-1201-4914-a181-d4da9349da98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.017338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bd4cee-1201-4914-a181-d4da9349da98 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.037832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed1e0c-ee29-42ed-9ec1-2f452505f673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.038638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1e0c-ee29-42ed-9ec1-2f452505f673 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.039140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1e0c-ee29-42ed-9ec1-2f452505f673 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.039648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed1e0c-ee29-42ed-9ec1-2f452505f673 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.040895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1e0c-ee29-42ed-9ec1-2f452505f673 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:01.041273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1e0c-ee29-42ed-9ec1-2f452505f673 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.042846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed1e0c-ee29-42ed-9ec1-2f452505f673 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.060625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 249/498] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:01.060625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e946e-f7a6-4d7c-a5f8-1fc6d8af0be9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.060625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e946e-f7a6-4d7c-a5f8-1fc6d8af0be9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.060625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e946e-f7a6-4d7c-a5f8-1fc6d8af0be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.082451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-9b5e946e-f7a6-4d7c-a5f8-1fc6d8af0be9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:01.083716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9b5e946e-f7a6-4d7c-a5f8-1fc6d8af0be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:01.115225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:01.126089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 250/499] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:01.130882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd4cee-1201-4914-a181-d4da9349da98 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.157294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:01.179283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 250/500] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:01.179283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e946e-f7a6-4d7c-a5f8-1fc6d8af0be9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.248366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bd4cee-1201-4914-a181-d4da9349da98 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.271505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e946e-f7a6-4d7c-a5f8-1fc6d8af0be9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.281040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.281715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.282002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.282274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.283034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:01.283375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.284771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.293396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 251/501] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:01.294122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.305974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.306781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.307064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.307368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.322280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.323956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:01.341831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:01.342833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 251/502] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:01.344026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.354258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:01.397971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:01.400110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105cd9a-a968-444f-b8f3-088804f235d1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.432832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28da009-dd39-46bb-a3df-a0344da37f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.433625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28da009-dd39-46bb-a3df-a0344da37f7a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.433900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28da009-dd39-46bb-a3df-a0344da37f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.434166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28da009-dd39-46bb-a3df-a0344da37f7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.443490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf6bdf-f720-4195-84bc-8915f70053ea None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.444313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28da009-dd39-46bb-a3df-a0344da37f7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.445793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28da009-dd39-46bb-a3df-a0344da37f7a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-b28da009-dd39-46bb-a3df-a0344da37f7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:01.462229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:01.463148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 252/503] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e3b4-b46c-4d0f-a171-272127188450 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e3b4-b46c-4d0f-a171-272127188450 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545e3b4-b46c-4d0f-a171-272127188450 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.469249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 253/504] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:01.469768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545e3b4-b46c-4d0f-a171-272127188450 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.479986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baf6be-7962-48ff-b88b-130233d94cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.480787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baf6be-7962-48ff-b88b-130233d94cdd None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.481069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baf6be-7962-48ff-b88b-130233d94cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.481353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75baf6be-7962-48ff-b88b-130233d94cdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.491918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75baf6be-7962-48ff-b88b-130233d94cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-75baf6be-7962-48ff-b88b-130233d94cdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:01.499651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:01.500722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:01.501515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 252/505] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:01.502184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baf6be-7962-48ff-b88b-130233d94cdd None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.576919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545e3b4-b46c-4d0f-a171-272127188450 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.584175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:01.598033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75baf6be-7962-48ff-b88b-130233d94cdd None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.610722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:01.611422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.612223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.612500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.612770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.622839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.624374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.631325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46878e3-091d-4ac1-8ad6-762e7f40345c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.631986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46878e3-091d-4ac1-8ad6-762e7f40345c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.632263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46878e3-091d-4ac1-8ad6-762e7f40345c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.632543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46878e3-091d-4ac1-8ad6-762e7f40345c None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.633376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46878e3-091d-4ac1-8ad6-762e7f40345c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:01.633683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46878e3-091d-4ac1-8ad6-762e7f40345c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.635111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46878e3-091d-4ac1-8ad6-762e7f40345c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:01.641659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:01.642710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:01.643612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:01.643612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:01.643612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:01.643612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:01.643612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 254/506] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:01.643612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.656881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 253/507] 10.222.0.69 () {56 vars in 1137 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:01.658120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f735ec5-394b-4bd6-b221-eb4d648130a5 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.658373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f735ec5-394b-4bd6-b221-eb4d648130a5 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.658671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f735ec5-394b-4bd6-b221-eb4d648130a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.674928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4f735ec5-394b-4bd6-b221-eb4d648130a5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:01.676114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4f735ec5-394b-4bd6-b221-eb4d648130a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:01.739637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 254/508] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:01.740458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f735ec5-394b-4bd6-b221-eb4d648130a5 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.743124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8be8a-94ab-4f27-a683-716700f9614d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.781339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.781997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.782296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.782566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.783297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:01.783609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.785002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.796537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 255/509] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:01.797307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.816610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:01.841791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:01.844672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f735ec5-394b-4bd6-b221-eb4d648130a5 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.878265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.878927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.879200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.879508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.880251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:01.880551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.881934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.889920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 255/510] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:01.890678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.898205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f373d31-5940-447d-aab1-297c75512a8f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:01.930641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:01.931333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:01.931620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:01.931894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:01.932632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:01.932939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:01.934319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:01.945881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 256/511] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:01.946632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:01.990866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f6087-b70f-4f26-8d24-d68e7282fc95 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.022255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.023690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a5b41-9640-474c-91b6-918a825e54f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.024450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a5b41-9640-474c-91b6-918a825e54f3 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.024738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a5b41-9640-474c-91b6-918a825e54f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.025011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19a5b41-9640-474c-91b6-918a825e54f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.044830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7b49b-6e8a-4144-a6ed-eb618c02c23e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.055475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.078200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.078986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.079284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.079559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.089196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.090628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.092972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a5b41-9640-474c-91b6-918a825e54f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.094444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19a5b41-9640-474c-91b6-918a825e54f3 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:02.107687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:02.108681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:02.109721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:02.109721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:02.109721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:02.109721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:02.109721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 257/512] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:02.110258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.196312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 256/513] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:02.196941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a5b41-9640-474c-91b6-918a825e54f3 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.212807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0dcc8d-8496-422b-a158-8d018e0868b1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.241089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.247462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.248254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.248536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.249068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.255459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.256043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.256934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:02.271829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:02.272710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:02.273666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 258/514] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:02.274136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.298818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19a5b41-9640-474c-91b6-918a825e54f3 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.332093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.332879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.333163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.333444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.343576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.344979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:02.369257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:02.370125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:02.370972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:02.370972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:02.370972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:02.370972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:02.370972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 257/515] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:02.371242 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.378426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c01312-bbb6-4205-a5c1-533d3d0c7a08 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.418083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.418906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.419184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.419481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.426456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.427990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:02.442621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:02.443591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:02.444484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 259/516] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:02.445246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.483044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548d750-c1b6-42be-99c6-6c2748392f33 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.496868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.497495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.517043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.517702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.517983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.518250 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.518966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:02.519281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.520661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.532437 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 258/517] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:02.533201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.549376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3634a8ad-2eec-4889-ae09-f608c83c7585 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.583813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.584599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.584880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.585147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.595022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.596454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:02.613154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:02.614006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:02.614865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:02.614865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:02.614865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:02.614865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:02.614865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 260/518] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:02.615123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.634308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c6721f-5ed0-41a2-8fdc-174c328513d8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.668661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.669318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.670317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.670317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.670636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:02.670944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.672446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.691164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 259/519] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:02.692008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.717915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39233f9f-c9e2-46be-b1b2-53c3058f0f26 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.719446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.729026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.751343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.752014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.752295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.752568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.753296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:02.753594 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.754983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.768263 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 261/520] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:02.768263 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.797878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c949f8-6df8-414d-a58f-c90ae53805b7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.829956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.830606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.830917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.831178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.832039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:02.832338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.833780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.850612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 260/521] 10.222.0.69 () {56 vars in 1139 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:02.851410 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.866383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2358ec08-56df-4ecb-817f-39ff446be636 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.898791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f411-a63b-4522-bbf7-5c22f93a134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.899496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f411-a63b-4522-bbf7-5c22f93a134a None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.899790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f411-a63b-4522-bbf7-5c22f93a134a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.900068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a4f411-a63b-4522-bbf7-5c22f93a134a None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.900780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4f411-a63b-4522-bbf7-5c22f93a134a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.902227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a4f411-a63b-4522-bbf7-5c22f93a134a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:02.923906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 262/522] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 09:00:02 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:02.924717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f411-a63b-4522-bbf7-5c22f93a134a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:02.947211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.953971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:02.956967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df71e8-9a67-486d-8c59-ae52f2a8846d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:02.994530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:02.995213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:02.995517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:02.995789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:02.996553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:02.996857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:02.998248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.010604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 261/523] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:03.011522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:03.052836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a4f411-a63b-4522-bbf7-5c22f93a134a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.091299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.092001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.092281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.092538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.093412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=04cd27bb637a404d827d384196eabfb1) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:03.093697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.095211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.102047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 263/524] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:02 2022] GET /identity/v3/role_assignments?user.id=04cd27bb637a404d827d384196eabfb1&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:03.102813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:03.137736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ead4d8-941d-478c-9c4a-f5ee73c7cfac None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.186023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.187506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.189732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.189732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.200648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.201404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:03.217081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:03.218118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:03.219005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 262/525] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:03.219591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:03.224204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:03.251711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:03.256413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1ef85-4948-4325-9932-b6435dc82217 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.294195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2de8b9-5995-4cd5-b286-777301dca826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.294870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2de8b9-5995-4cd5-b286-777301dca826 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.295152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2de8b9-5995-4cd5-b286-777301dca826 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.295451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2de8b9-5995-4cd5-b286-777301dca826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.359845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee14a84-5978-4103-a092-1ae138a8d718 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.380004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2de8b9-5995-4cd5-b286-777301dca826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.381647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2de8b9-5995-4cd5-b286-777301dca826 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.403757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.405101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.405526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.405937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.416112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.418438 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:03.438319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:03.439501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:03.440355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:03.440355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:03.440355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:03.440355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:03.440355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:03.440355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:03.440355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:03.440355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:03.447688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 263/526] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:03.448930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:03.472180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:03.494320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 264/527] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:03.495166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2de8b9-5995-4cd5-b286-777301dca826 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:03.526909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:03.565418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a717b-3357-4f55-8ee9-2887f9f9f4e2 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.601548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.602394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.602787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.603156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.604120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:03.604518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.606055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.616926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2de8b9-5995-4cd5-b286-777301dca826 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.624712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 264/528] 10.222.0.69 () {56 vars in 1137 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:03.625622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:03.658097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.659004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.659707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.659986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.661054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:03.661606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.663817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.686765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 265/529] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:03.687890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:03.733678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:03.744455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89858289-995c-447d-a33c-716c9567bb91 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.778449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:03.785563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.786451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.786731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.787002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.798830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.800248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.808933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f6f0b3-ef05-42ef-9bd4-bafd9d71bdbd None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:03.819542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:03.820689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:03.821714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:03.821714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:03.821714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:03.821714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:03.825386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 265/530] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:03.826300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:03.851988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b988ab-2144-4bb7-b49b-60259da7bd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.853135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b988ab-2144-4bb7-b49b-60259da7bd04 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.853966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b988ab-2144-4bb7-b49b-60259da7bd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.854544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b988ab-2144-4bb7-b49b-60259da7bd04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.932966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9fd9e-e28f-4f3b-9747-674d155ee2c4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:03.947999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b988ab-2144-4bb7-b49b-60259da7bd04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.949931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b988ab-2144-4bb7-b49b-60259da7bd04 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.968132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:03.968974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:03.969457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:03.969865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:03.970885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=4b672a2898b849338844b1c8e911646d) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:03.971321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:03.972838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:03.978913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 266/531] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/role_assignments?user.id=4b672a2898b849338844b1c8e911646d&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:03.979890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:04.003358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:04.039072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:04.071402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 266/532] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:04.073643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b988ab-2144-4bb7-b49b-60259da7bd04 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:04.108884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d558d1-a5a6-4476-9866-2c38bbd19f16 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:04.147844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:04.148781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:04.149153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:04.149515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:04.150565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:04.150837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:04.152340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:04.170911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 267/533] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:04.172434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:04.188561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b988ab-2144-4bb7-b49b-60259da7bd04 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:04.274374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:04.275196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:04.278158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:04.278158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:04.282791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2bfaf8-cb8c-43a5-99c5-d4832f4eb79e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:04.301973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:04.302258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:04.304192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:04.309447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:04.329407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:04.330491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:04.331577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:04.331577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:04.331577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:04.331577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:04.337544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 267/534] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:04.338982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:04.366058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:04.367520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:04.367941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:04.368344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:04.385888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:04.387668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:04.407734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:04.408804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:04.409844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:04.409844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:04.409844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:04.409844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:04.409844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:04.409844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:04.409844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:04.409844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:04.421236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 268/535] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:04.434243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:04.491119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761eedd6-7d0f-4675-ac9d-fbced6a8d6a0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:04.572181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec1c1a-98ed-4b2a-9377-7c3819be6922 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:04.576978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:04.577672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:04.577959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:04.578237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:04.578983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:04.579317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:04.580722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:04.606268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:04.606985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:04.607286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:04.607574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:04.608426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:04.608737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:04.610198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:04.617353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 268/536] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:04.628256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 269/537] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:04.628427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:04.635485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:04.690051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:04.711621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:04.749921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb59daf1-1029-4680-a387-c00600983a50 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:04.813553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:04.814359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:04.814645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:04.814926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:04.833768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:04.835279 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:04.854135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:04.855012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:04.856469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 269/538] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:04.861013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:04.864132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae6e24-a4a2-4d37-880e-34f627fea93e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:04.925402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:04.926071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:04.926351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:04.926622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:04.927504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=349e99511c7c4307b27dc90b01f2dcd3) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:04.927809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:04.929318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:04.943830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 270/539] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:04 2022] GET /identity/v3/role_assignments?user.id=349e99511c7c4307b27dc90b01f2dcd3&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 259 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:05.034834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43ba9df-d20e-4970-be0b-e06ef1fd6eed None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:05.058969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:05.092176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9562944-9ec6-48d9-a6af-32133c327ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:05.092866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562944-9ec6-48d9-a6af-32133c327ace None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:05.093149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562944-9ec6-48d9-a6af-32133c327ace None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:05.093427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9562944-9ec6-48d9-a6af-32133c327ace None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:05.094314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9562944-9ec6-48d9-a6af-32133c327ace None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:05.094631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9562944-9ec6-48d9-a6af-32133c327ace None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:05.096174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9562944-9ec6-48d9-a6af-32133c327ace None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:05.117163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 270/540] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:05.122819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:05.274937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:05.286380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49115e61-06ef-4247-8cf8-36721037f88c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:05.334616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb814a6-20da-425c-9111-73e692115a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:05.335374 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb814a6-20da-425c-9111-73e692115a51 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:05.335690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb814a6-20da-425c-9111-73e692115a51 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:05.335963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb814a6-20da-425c-9111-73e692115a51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:05.336861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb814a6-20da-425c-9111-73e692115a51 None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=68d25a45f9ee4ae58c6751a8781d12d1) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:05.337173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb814a6-20da-425c-9111-73e692115a51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:05.338613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb814a6-20da-425c-9111-73e692115a51 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:05.344962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 271/541] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:05 2022] GET /identity/v3/role_assignments?user.id=68d25a45f9ee4ae58c6751a8781d12d1&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:05.376523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd500091-a39d-4098-9176-2cb62708a1d4 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:05.377047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd500091-a39d-4098-9176-2cb62708a1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:05.377047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd500091-a39d-4098-9176-2cb62708a1d4 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:05.378794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 271/542] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:05.384801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0e9e-dda8-4bef-8a78-f59390b2631f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:05.385269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0e9e-dda8-4bef-8a78-f59390b2631f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:05.385368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf0e9e-dda8-4bef-8a78-f59390b2631f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:05.403494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4ecf0e9e-dda8-4bef-8a78-f59390b2631f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:05.405024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4ecf0e9e-dda8-4bef-8a78-f59390b2631f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:05.475130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 272/543] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:05.499333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd500091-a39d-4098-9176-2cb62708a1d4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:05.640369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd500091-a39d-4098-9176-2cb62708a1d4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:05.685271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10980b-fec5-4682-ab92-fb0a5f9fffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:05.687900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10980b-fec5-4682-ab92-fb0a5f9fffbe None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:05.692407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10980b-fec5-4682-ab92-fb0a5f9fffbe None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:05.693024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10980b-fec5-4682-ab92-fb0a5f9fffbe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:05.711849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10980b-fec5-4682-ab92-fb0a5f9fffbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-6a10980b-fec5-4682-ab92-fb0a5f9fffbe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:05.720063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:05.720972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:05.721764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:05.725509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 272/544] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:05.731748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0e9e-dda8-4bef-8a78-f59390b2631f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:05.828042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:05.844545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf0e9e-dda8-4bef-8a78-f59390b2631f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:05.882952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358a2d-1ccb-4bb9-b606-293f157effa5 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:05.883291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358a2d-1ccb-4bb9-b606-293f157effa5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:05.883538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8358a2d-1ccb-4bb9-b606-293f157effa5 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:05.884923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 273/545] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:05.891075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46b2a4-3401-4a88-9f8f-ace9cf49ad3d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:05.891355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46b2a4-3401-4a88-9f8f-ace9cf49ad3d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:05.891610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e46b2a4-3401-4a88-9f8f-ace9cf49ad3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:05.897408 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:05.898159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:05.898439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:05.898712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:05.899497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:05.899801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:05.901168 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:05.909810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 273/546] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:05.915651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:05.932878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6e46b2a4-3401-4a88-9f8f-ace9cf49ad3d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:05.934090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6e46b2a4-3401-4a88-9f8f-ace9cf49ad3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:05.999968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:06.060657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 274/547] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:06.087473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6421ee6d-5aa7-4db5-9d6b-b39e7256f1c0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:06.087675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46b2a4-3401-4a88-9f8f-ace9cf49ad3d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:06.125105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:06.126461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.127723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.130911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.148723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:06.151052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:06.171650 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:06.182451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:06.183489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:06.186375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:06.186375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:06.186375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:06.186375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:06.189100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 274/548] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:06.196067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:06.262127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e46b2a4-3401-4a88-9f8f-ace9cf49ad3d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:06.329221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592b02-4124-4c3d-9247-4f6783481b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:06.330039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592b02-4124-4c3d-9247-4f6783481b9f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.330322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592b02-4124-4c3d-9247-4f6783481b9f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.330595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd592b02-4124-4c3d-9247-4f6783481b9f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.354959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd592b02-4124-4c3d-9247-4f6783481b9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-fd592b02-4124-4c3d-9247-4f6783481b9f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:06.367114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:06.368465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:06.369360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:06.381337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 275/549] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:06.383747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592b02-4124-4c3d-9247-4f6783481b9f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:06.402191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db58a68f-5430-44c0-9d47-e70360faf9a1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:06.442463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:06.459380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:06.460051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.460530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.460868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.461603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:06.461952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:06.463363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:06.484536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 275/550] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:06.490519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:06.554739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:06.577233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd592b02-4124-4c3d-9247-4f6783481b9f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:06.604563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab1ac9f-3119-44ff-ba36-1bcf0c52f832 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:06.610626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062d58df-3059-4314-965f-6d96a65855b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:06.611320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d58df-3059-4314-965f-6d96a65855b9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.611604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d58df-3059-4314-965f-6d96a65855b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.611879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062d58df-3059-4314-965f-6d96a65855b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.612637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d58df-3059-4314-965f-6d96a65855b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:06.612942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d58df-3059-4314-965f-6d96a65855b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:06.614334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062d58df-3059-4314-965f-6d96a65855b9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:06.622865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 276/551] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:06.624239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca3ca8-a8f0-4e46-844e-05fe6feaba67 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.624528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca3ca8-a8f0-4e46-844e-05fe6feaba67 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.624804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ca3ca8-a8f0-4e46-844e-05fe6feaba67 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.626284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 277/552] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:06.627959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca3ca8-a8f0-4e46-844e-05fe6feaba67 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:06.643921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092392ef-01a6-4c76-8f17-b245e0cf09a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:06.644770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092392ef-01a6-4c76-8f17-b245e0cf09a1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.645054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092392ef-01a6-4c76-8f17-b245e0cf09a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.645326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092392ef-01a6-4c76-8f17-b245e0cf09a1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.653315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092392ef-01a6-4c76-8f17-b245e0cf09a1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:06.654859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092392ef-01a6-4c76-8f17-b245e0cf09a1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-092392ef-01a6-4c76-8f17-b245e0cf09a1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:06.670885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:06.672027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:06.672942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 276/553] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:06.674012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b5bf7-f420-4edc-89aa-42665c067f7a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.674174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b5bf7-f420-4edc-89aa-42665c067f7a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.676329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2b5bf7-f420-4edc-89aa-42665c067f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.696427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5f2b5bf7-f420-4edc-89aa-42665c067f7a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:06.697630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5f2b5bf7-f420-4edc-89aa-42665c067f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:06.715732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:06.732516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ca3ca8-a8f0-4e46-844e-05fe6feaba67 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:06.786584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:06.786898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 277/554] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:06.787415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.787698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.787978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.788380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b5bf7-f420-4edc-89aa-42665c067f7a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:06.810085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:06.813272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:06.836596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:06.837774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:06.838831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:06.838831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:06.838831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:06.838831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:06.839547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 278/555] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:06.840413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:06.875665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:06.923822 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2b5bf7-f420-4edc-89aa-42665c067f7a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:06.955585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbd5a0-0df4-4956-9588-324e0b548af8 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:06.959740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:06.960461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.960741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:06.961013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:06.961847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:06.962148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:06.963620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:06.988843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 278/556] 10.222.0.69 () {56 vars in 1135 bytes} [Fri May 20 09:00:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:06.988843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:06.998581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cc2e2-ca48-4082-9cc8-e735fc49531c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:06.999449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cc2e2-ca48-4082-9cc8-e735fc49531c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:06.999739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cc2e2-ca48-4082-9cc8-e735fc49531c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.000002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294cc2e2-ca48-4082-9cc8-e735fc49531c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.007034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:07.010902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294cc2e2-ca48-4082-9cc8-e735fc49531c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-294cc2e2-ca48-4082-9cc8-e735fc49531c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:07.018676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:07.019592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:07.020977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 279/557] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:07.021664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cc2e2-ca48-4082-9cc8-e735fc49531c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.102738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb22ec-c3df-4b41-ae16-ff5266806b9e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.111046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:07.123571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294cc2e2-ca48-4082-9cc8-e735fc49531c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.141480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.142383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.142803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.143237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.144626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:07.145077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.147081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.158757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0272-cc23-4024-9223-ad802ca87e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.159532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0272-cc23-4024-9223-ad802ca87e5e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.160005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0272-cc23-4024-9223-ad802ca87e5e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.160350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e0272-cc23-4024-9223-ad802ca87e5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.167158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 279/558] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:07.167158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.232791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:07.236696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0272-cc23-4024-9223-ad802ca87e5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.238427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e0272-cc23-4024-9223-ad802ca87e5e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.301021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca53dc7f-b589-4d9d-b66d-0783c376393c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.336778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.337445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.337768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.338040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.338952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:07.339277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.340967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.351817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 280/559] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:07.352815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.363747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 280/560] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:07.365961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0272-cc23-4024-9223-ad802ca87e5e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.387094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:07.487491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b97748c-a156-4aaf-ae4d-6a91af386e0a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.503894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:07.509925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e0272-cc23-4024-9223-ad802ca87e5e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.527156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.528338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.528866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.529345 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.537792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.539786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.556030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.556905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.557183 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.557452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:07.558492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:07.559665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:07.560622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:07.567611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 281/561] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:07.568552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.568904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.570119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:07.593317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:07.594282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:07.595164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:07.595164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:07.595164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:07.595164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:07.595164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 281/562] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:07.595481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.685756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:07.699726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec9f50-5ce2-41fb-b50f-f6589164c010 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.699969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd6463-86ca-431f-b111-24625f81dcfb None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.740466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368986d6-f682-4a32-a4da-9035277fe585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.740673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.741053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368986d6-f682-4a32-a4da-9035277fe585 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.741338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368986d6-f682-4a32-a4da-9035277fe585 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.741579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.741700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368986d6-f682-4a32-a4da-9035277fe585 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.742080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.742436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.742525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368986d6-f682-4a32-a4da-9035277fe585 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:07.742826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368986d6-f682-4a32-a4da-9035277fe585 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.744304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368986d6-f682-4a32-a4da-9035277fe585 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.753749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.755340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:07.773768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:07.774923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:07.776122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:07.776122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:07.776122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:07.776122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:07.779436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 282/563] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:07.780269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.786571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 282/564] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:07.788345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c197f-e364-4834-9817-c230fe621e45 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.788881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c197f-e364-4834-9817-c230fe621e45 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.789363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c197f-e364-4834-9817-c230fe621e45 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.791856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 283/565] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:07.793221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c197f-e364-4834-9817-c230fe621e45 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.807328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:07.881813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473fef9-fa18-4237-9414-241746d717ee None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.903057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c197f-e364-4834-9817-c230fe621e45 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:07.917115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.917784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.918061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.918332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.919055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:07.919391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.920766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.929270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:07.936163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15536cf9-0cd0-474b-9889-e83522499110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:07.936831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15536cf9-0cd0-474b-9889-e83522499110 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.937115 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15536cf9-0cd0-474b-9889-e83522499110 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.937226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 283/566] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:07.937391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15536cf9-0cd0-474b-9889-e83522499110 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.938040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:07.938131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15536cf9-0cd0-474b-9889-e83522499110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:07.938424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15536cf9-0cd0-474b-9889-e83522499110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:07.939847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15536cf9-0cd0-474b-9889-e83522499110 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:07.951053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 284/567] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:07.952288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1f91-19b4-4bbc-a207-d255c557195a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:07.952580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1f91-19b4-4bbc-a207-d255c557195a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:07.953043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1f91-19b4-4bbc-a207-d255c557195a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:07.969956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-2e5e1f91-19b4-4bbc-a207-d255c557195a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:07.971149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2e5e1f91-19b4-4bbc-a207-d255c557195a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:08.033349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:08.039093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cfd93-a16d-455b-8905-f849197b591c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.039674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 285/568] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:08.041418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce390653-e4ad-4bb9-81a4-166b8857f3f2 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.041683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce390653-e4ad-4bb9-81a4-166b8857f3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.042583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce390653-e4ad-4bb9-81a4-166b8857f3f2 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.047250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 286/569] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:08.047250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce390653-e4ad-4bb9-81a4-166b8857f3f2 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.076943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.077745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.078025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.078295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.088358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.089835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:08.106945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:08.108096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:08.109028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:08.109028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:08.109028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:08.109028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:08.109028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 284/570] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:08.109028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.153525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce390653-e4ad-4bb9-81a4-166b8857f3f2 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.161506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:08.188080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f13ab-e1e5-4326-a016-266eb4a1a06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.188910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f13ab-e1e5-4326-a016-266eb4a1a06c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.189199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f13ab-e1e5-4326-a016-266eb4a1a06c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.189475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f13ab-e1e5-4326-a016-266eb4a1a06c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.197048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f13ab-e1e5-4326-a016-266eb4a1a06c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.198537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f13ab-e1e5-4326-a016-266eb4a1a06c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.208263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1aad8-7299-4032-bc69-fc2749600f18 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d18f13ab-e1e5-4326-a016-266eb4a1a06c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:08.213157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:08.214265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:08.215252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 287/571] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:08.216245 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377cce6-9648-4c0a-99e1-6d1fa1f38d7f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.216342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377cce6-9648-4c0a-99e1-6d1fa1f38d7f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.216580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3377cce6-9648-4c0a-99e1-6d1fa1f38d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.233195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3377cce6-9648-4c0a-99e1-6d1fa1f38d7f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:08.234391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3377cce6-9648-4c0a-99e1-6d1fa1f38d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:08.243290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.244099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.244377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.244651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.251289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.252753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:08.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:08.267790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 285/572] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:08.268693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.291326 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:08.306143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 288/573] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:08.307035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377cce6-9648-4c0a-99e1-6d1fa1f38d7f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.370718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee68de4-c086-4b3d-af74-c8f83d2bc10b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.384072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:08.403857 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3377cce6-9648-4c0a-99e1-6d1fa1f38d7f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.405567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d88e5-d616-4321-b75e-452001ff79fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.406361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d88e5-d616-4321-b75e-452001ff79fd None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.406636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d88e5-d616-4321-b75e-452001ff79fd None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.406905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d88e5-d616-4321-b75e-452001ff79fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.417151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d88e5-d616-4321-b75e-452001ff79fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-ae2d88e5-d616-4321-b75e-452001ff79fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:08.424624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:08.425720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 286/574] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:08.426587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d88e5-d616-4321-b75e-452001ff79fd None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.442973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.443833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.444116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.444391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.445182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:08.445490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.446897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.458400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 289/575] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:08.459175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.509380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:08.538727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d88e5-d616-4321-b75e-452001ff79fd None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.561066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fcd73f-44d8-40cf-a050-aa31abe2d1b8 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.573170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.573822 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.574103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.574373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.575213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:08.575566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.577017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.593621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 287/576] 10.222.0.69 () {56 vars in 1137 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:08.594420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.594574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.595551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.595657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.595958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.596850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:08.597049 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.598767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.616246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 290/577] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:08.617643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.625299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:08.696055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc75296-1639-46da-909d-97c81c1ae894 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.722879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5269c0-0127-4a18-b736-9d626fd8b970 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.729136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7fcf-0ab8-4def-9996-b1732d296db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.729923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7fcf-0ab8-4def-9996-b1732d296db6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.730222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7fcf-0ab8-4def-9996-b1732d296db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.730500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e7fcf-0ab8-4def-9996-b1732d296db6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.732001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:08.740519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e7fcf-0ab8-4def-9996-b1732d296db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-ed6e7fcf-0ab8-4def-9996-b1732d296db6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:08.748112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:08.749207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:08.750218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 288/578] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:08.751031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7fcf-0ab8-4def-9996-b1732d296db6 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.760051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.760756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.761030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.761297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.762044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:08.762348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.763775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.773700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 291/579] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:08.774513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.843890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:08.853856 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e7fcf-0ab8-4def-9996-b1732d296db6 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.868951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebfeae4-6294-4477-9572-8d8250ad4e5c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:08.886891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.887719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.887999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.888621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.894949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.898619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.902621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:08.903450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:08.903726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:08.903991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:08.912602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:08.913586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:08.914837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:08.914837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:08.914837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:08.914837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:08.914837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:08.914837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:08.914837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:08.914837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:08.915794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:08.915866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 289/580] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:08.915866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.915986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:08.935404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:08.936299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:08.937334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:08.937334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:08.937334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:08.937334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:08.937334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 292/581] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:08.937607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:08.961878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.013285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140f8d2-08da-4dbd-96d2-6a9e3f42f376 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.036656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75951240-286a-45b8-91f7-e054eeac0aa0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.045163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.045806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.046079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.046351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.047212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=0c8203a8a7464c3bac93e5d42808c00f) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:09.047547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.048966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.054216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 290/582] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/role_assignments?user.id=0c8203a8a7464c3bac93e5d42808c00f&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:09.054969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.070201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.070894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.071174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.071475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.072265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:09.072574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.074012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.083289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 293/583] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:09.083973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.089213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.176362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.185682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae144c-2662-4eb1-9ba8-a02690154795 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.196984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54599dc-f715-439b-9f47-e283280ce3b1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.223139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.224016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.224293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.224560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.233639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.234324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.234608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.234891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.235761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:09.236075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.237532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.237646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.239305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.251539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 294/584] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:09.252614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:09.259020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:09.265725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:09.266623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:09.266623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:09.266623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:09.266623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:09.266623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 291/585] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:09.266623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.330032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.359470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e3b56-3e08-4dbc-a2cf-1326c9151bcc None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.365845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcb00ba-a6db-48c0-9b60-5fa060279d98 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.393918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.394585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.394861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.395136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.395887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:09.396194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.397577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.399329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.400133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.400414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.400688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.413840 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 295/586] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:09.414102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.415516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.415839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.420407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:09.435716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:09.436714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:09.437661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:09.437661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:09.437661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:09.437661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:09.437661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 292/587] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:09.437661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.526005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1e2af8-fdef-42f6-bbee-230bcbea84b4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.545956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e6011-b076-47df-989c-c17d8eaa15e0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.561015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b55b5a-b419-4e6b-81ee-faceaee87ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.561915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55b5a-b419-4e6b-81ee-faceaee87ffe None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.562166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55b5a-b419-4e6b-81ee-faceaee87ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.562464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b55b5a-b419-4e6b-81ee-faceaee87ffe None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/04cd27bb637a404d827d384196eabfb1/roles/99ad2a72e54b4e51baabe132c647c8e3` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.567193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.577999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b55b5a-b419-4e6b-81ee-faceaee87ffe None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=04cd27bb637a404d827d384196eabfb1, role_id=99ad2a72e54b4e51baabe132c647c8e3)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.579488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b55b5a-b419-4e6b-81ee-faceaee87ffe None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.586901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.587591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.587892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.588134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.588855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:09.589152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.590505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.593944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 296/588] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:09 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/04cd27bb637a404d827d384196eabfb1/roles/99ad2a72e54b4e51baabe132c647c8e3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:00:09.594860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b55b5a-b419-4e6b-81ee-faceaee87ffe None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.622501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 293/589] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:09.623412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.689940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.736841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fadda-b43c-4685-b34f-dce358d45aa4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.750404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b55b5a-b419-4e6b-81ee-faceaee87ffe None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.776457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.777355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.777901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.778266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.779167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:09.779587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.781221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.799500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 294/590] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:09.799500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.800640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.801653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.802097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.802519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.810387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.812270 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:09.833361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:09.834466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:09.835648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 297/591] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:09.836273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:09.863091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.943599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27b8db-49f5-4d99-bfce-67c7cede93b9 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.950231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976282f7-fac4-4a1a-bef4-2d95e0920c59 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:09.967658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:09.980083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.981221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.981673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.982237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.985181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:09.986435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:09.986435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:09.986766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:09.991638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.993398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:09.995459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:09.996956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:10.010147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:10.011361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 20 09:00:10.012337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:10.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:10.013751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 298/592] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:10.014689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.017337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 295/593] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:10.022543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.115244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764749b8-b9a2-4999-9913-32633739f93a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:10.123344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:10.127972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b19723-750f-4057-8ffc-d2b24fc88e83 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:10.150097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc39364-271c-45ea-9247-165916d1743c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:10.150940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc39364-271c-45ea-9247-165916d1743c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:10.151356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc39364-271c-45ea-9247-165916d1743c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:10.152014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc39364-271c-45ea-9247-165916d1743c None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:10.152613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc39364-271c-45ea-9247-165916d1743c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:10.153014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc39364-271c-45ea-9247-165916d1743c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:10.154593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc39364-271c-45ea-9247-165916d1743c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.168370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:10.169420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:10.169880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:10.170322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:10.171411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:10.171911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:10.173476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 299/594] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:10.173961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.174583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc39364-271c-45ea-9247-165916d1743c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.194979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 296/595] 10.222.0.69 () {56 vars in 1137 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:10.196190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.215630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:10.275736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc39364-271c-45ea-9247-165916d1743c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:10.300566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cc0716-128b-4a79-859e-d6a57634995f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:10.310388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:10.311148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:10.311520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:10.311856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:10.312768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:10.313138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:10.314647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.332361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 300/596] 10.222.0.69 () {56 vars in 1139 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:10.333334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.348878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba687e1-5a68-4e6e-8f27-d7dae805f26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:10.349952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba687e1-5a68-4e6e-8f27-d7dae805f26e None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:10.350301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba687e1-5a68-4e6e-8f27-d7dae805f26e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:10.350646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba687e1-5a68-4e6e-8f27-d7dae805f26e None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/4b672a2898b849338844b1c8e911646d/roles/99ad2a72e54b4e51baabe132c647c8e3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:10.367159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba687e1-5a68-4e6e-8f27-d7dae805f26e None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=4b672a2898b849338844b1c8e911646d, role_id=99ad2a72e54b4e51baabe132c647c8e3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:10.368945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba687e1-5a68-4e6e-8f27-d7dae805f26e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.388209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 297/597] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:10 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/4b672a2898b849338844b1c8e911646d/roles/99ad2a72e54b4e51baabe132c647c8e3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:00:10.389173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba687e1-5a68-4e6e-8f27-d7dae805f26e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.407194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:10.455840 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:10.466059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a4da0e-d1f5-493a-9f06-f135f88e5c11 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:10.515140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:10.516006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:10.516285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:10.516562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:10.547220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:10.548822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:10.588906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:10.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:10.593090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:10.593090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:10.593090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:10.593090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:10.595856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 301/598] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:10.602531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.620423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba687e1-5a68-4e6e-8f27-d7dae805f26e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:10.686755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:10.688152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:10.688444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:10.688719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:10.704138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:10.706083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:10.729275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:10.731274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:10.733296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:10.733296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:10.733296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:10.733296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:10.733296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 298/599] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:10.740356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.754005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f0095c-4bc9-4c38-b40f-5a845836603f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:10.805076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:10.805993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:10.806280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:10.806959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:10.807987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:10.808296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:10.809009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:10.810624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.827845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 302/600] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:10.836248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:10.839992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:10.858363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f377426-9489-4614-94b4-55d4dc02a7ad None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:10.898194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:10.898864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:10.899145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:10.899451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:10.900187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:10.900493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:10.901951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:10.917468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 299/601] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:10.923489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:11.011142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1db06-1a8a-49a0-9553-396478036666 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:11.036192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158cc679-cb8f-47f8-9965-1b587397be28 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:11.045812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:11.046718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.047001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.047293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.064089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:11.065305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:11.066787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:11.073217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:11.074070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.074397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.074676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.091812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:11.093283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:11.096580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:11.097424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:11.098282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:11.098282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:11.098282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:11.098282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:11.098282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:11.098282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:11.098282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:11.098282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:11.102398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 303/602] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:11.104169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:11.113021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:11.113901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:11.114783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 300/603] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:11.119780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:11.227812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:11.270623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2702c7e7-20f3-4ca8-9d61-10c1a3ebe347 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:11.304712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:11.305376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.305653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.305972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.306909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:11.307216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:11.308890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:11.316716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e0b822-65f3-4887-969d-f01833e6515c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:11.332235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 304/604] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:11.337749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:11.369305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:11.370283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.370809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.371038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.371866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:11.372170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:11.373601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:11.405862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 301/605] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:11.413073 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:11.478172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117d48f-9da9-4cb0-a610-691cd9bd7a9b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:11.482371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:11.504679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:11.515869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371f0543-554d-4909-89c9-748efef476f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:11.516774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f0543-554d-4909-89c9-748efef476f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.517055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f0543-554d-4909-89c9-748efef476f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.517331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371f0543-554d-4909-89c9-748efef476f1 None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/68d25a45f9ee4ae58c6751a8781d12d1/roles/99ad2a72e54b4e51baabe132c647c8e3` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.533348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f0543-554d-4909-89c9-748efef476f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=68d25a45f9ee4ae58c6751a8781d12d1, role_id=99ad2a72e54b4e51baabe132c647c8e3)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:11.534760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371f0543-554d-4909-89c9-748efef476f1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:11.554338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c095ef1-86dd-4370-9518-084902c1b1af None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:11.559467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 305/606] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:11 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/68d25a45f9ee4ae58c6751a8781d12d1/roles/99ad2a72e54b4e51baabe132c647c8e3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:00:11.562897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50948-002e-4668-bd27-07f0eeb7e48e None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.564618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50948-002e-4668-bd27-07f0eeb7e48e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.565541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde50948-002e-4668-bd27-07f0eeb7e48e None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.570084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 306/607] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:11 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:11.577182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820512e-6624-485e-8d29-b257ec71c4fd None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.577818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820512e-6624-485e-8d29-b257ec71c4fd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.578307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820512e-6624-485e-8d29-b257ec71c4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.610275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a820512e-6624-485e-8d29-b257ec71c4fd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:11.613675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a820512e-6624-485e-8d29-b257ec71c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:11.634282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f406f20-a92c-4f45-80f3-ce6729f9bcf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:11.638143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406f20-a92c-4f45-80f3-ce6729f9bcf7 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.638748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406f20-a92c-4f45-80f3-ce6729f9bcf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.639387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f406f20-a92c-4f45-80f3-ce6729f9bcf7 None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/349e99511c7c4307b27dc90b01f2dcd3/roles/99ad2a72e54b4e51baabe132c647c8e3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.670405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f406f20-a92c-4f45-80f3-ce6729f9bcf7 None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=349e99511c7c4307b27dc90b01f2dcd3, role_id=99ad2a72e54b4e51baabe132c647c8e3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:11.672516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f406f20-a92c-4f45-80f3-ce6729f9bcf7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:11.698121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 302/608] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:11 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/349e99511c7c4307b27dc90b01f2dcd3/roles/99ad2a72e54b4e51baabe132c647c8e3 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:00:11.748282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 307/609] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 09:00:11.789512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f406f20-a92c-4f45-80f3-ce6729f9bcf7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:11.904271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:11.918720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f406f20-a92c-4f45-80f3-ce6729f9bcf7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:11.963190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b520507-5d64-4249-95df-7868c987e4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:11.964048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b520507-5d64-4249-95df-7868c987e4b8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:11.964325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b520507-5d64-4249-95df-7868c987e4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:11.964604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b520507-5d64-4249-95df-7868c987e4b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:11.975729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b520507-5d64-4249-95df-7868c987e4b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-5b520507-5d64-4249-95df-7868c987e4b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:11.983675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:11.984640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:11.985475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 303/610] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:11.991206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820512e-6624-485e-8d29-b257ec71c4fd None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:12.096866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820512e-6624-485e-8d29-b257ec71c4fd None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:12.099150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:12.119304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd56b-a7a6-48fc-8c22-388f9d3b365a None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.119647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd56b-a7a6-48fc-8c22-388f9d3b365a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.119848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedd56b-a7a6-48fc-8c22-388f9d3b365a None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.121477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 304/611] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:12.132600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bd681-3d5c-444f-8b2d-7d5735dad782 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.133015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bd681-3d5c-444f-8b2d-7d5735dad782 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.133393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196bd681-3d5c-444f-8b2d-7d5735dad782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.144090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:12.144798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.145092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.145380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.146158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:12.146471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:12.148153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:12.153461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-196bd681-3d5c-444f-8b2d-7d5735dad782 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:12.155056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-196bd681-3d5c-444f-8b2d-7d5735dad782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:12.161934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 308/612] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:12.163639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:12.226030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 305/613] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:12.262843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bd681-3d5c-444f-8b2d-7d5735dad782 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:12.270623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375c65b7-d606-4e98-b09e-d8a0b5688f0f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:12.314496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:12.315750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.316949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.317987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.339864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:12.341828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:12.382323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:12.383790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:12.384884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:12.386014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:12.386014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:12.386014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:12.386014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:12.388414 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 309/614] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:12.400248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:12.422175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:12.425806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196bd681-3d5c-444f-8b2d-7d5735dad782 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:12.514739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a959b-4b96-4c90-987b-794a0db96784 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:12.523976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f10c5-987c-4300-8f49-01ab47d6af65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:12.524876 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f10c5-987c-4300-8f49-01ab47d6af65 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.525392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f10c5-987c-4300-8f49-01ab47d6af65 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.525796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f10c5-987c-4300-8f49-01ab47d6af65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.552510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:12.553438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.553898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.554351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.555231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:12.555722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:12.557250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:12.560177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f10c5-987c-4300-8f49-01ab47d6af65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:12.574512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 310/615] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-fb4f10c5-987c-4300-8f49-01ab47d6af65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:12.575889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:12.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:12.578142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 306/616] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:12.579661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:12.582857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f10c5-987c-4300-8f49-01ab47d6af65 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:12.663684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:12.688653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1909234b-6c3e-4ac8-8f91-5967492fce13 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:12.750285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:12.751097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.751402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.751677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.765463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:12.766955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:12.792780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f10c5-987c-4300-8f49-01ab47d6af65 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:12.795197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:12.796342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:12.797620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 311/617] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:12.804124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:12.868853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:12.869530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.869869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.870148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.870958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:12.871281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:12.872672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:12.883587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:12.891213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 307/618] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:12.896254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:12.921571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790f073-dd6f-489f-8f3c-797d8d121bf7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:12.957584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:12.958259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:12.958540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:12.958813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:12.959685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:12.960000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:12.961523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:12.969501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:12.980226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 312/619] 10.222.0.69 () {56 vars in 1135 bytes} [Fri May 20 09:00:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:12.985660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.088356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a0994-31da-4079-a6f7-a5b27033a78d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.088917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7065934-aad9-4380-ba86-48f14af0c541 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.123389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:13.124400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.124782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.125150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.136801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:13.138404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:13.158891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:13.159924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:13.160983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:13.160983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:13.160983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:13.160983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:13.160983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 313/620] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:13.167710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.176586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17f634b-770c-4adc-aea6-9906d80d9dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:13.177386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f634b-770c-4adc-aea6-9906d80d9dea None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.177666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f634b-770c-4adc-aea6-9906d80d9dea None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.177934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17f634b-770c-4adc-aea6-9906d80d9dea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.199588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f634b-770c-4adc-aea6-9906d80d9dea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:13.203384 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17f634b-770c-4adc-aea6-9906d80d9dea None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-c17f634b-770c-4adc-aea6-9906d80d9dea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:13.226772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:13.227773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:13.228667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:13.228667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:13.228667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:13.228667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:13.231065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 308/621] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:13.232746 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20898721-0cd2-4ee7-b146-df9e6828d6c7 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.233292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20898721-0cd2-4ee7-b146-df9e6828d6c7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.233790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20898721-0cd2-4ee7-b146-df9e6828d6c7 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.239450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 309/622] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:13.240039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:13.242107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20898721-0cd2-4ee7-b146-df9e6828d6c7 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.297512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f357d5ea-42ac-41db-b4ca-519dc233d3bf None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.302455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:13.332052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bde3b-af33-4285-89f1-11a8429ae840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:13.332716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bde3b-af33-4285-89f1-11a8429ae840 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.332997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bde3b-af33-4285-89f1-11a8429ae840 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.333270 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bde3b-af33-4285-89f1-11a8429ae840 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.334004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bde3b-af33-4285-89f1-11a8429ae840 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:13.334309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bde3b-af33-4285-89f1-11a8429ae840 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:13.335774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080bde3b-af33-4285-89f1-11a8429ae840 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:13.354269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 314/623] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:13.354269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad03f-6d3c-4811-ba78-03dca3e7da62 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.354269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad03f-6d3c-4811-ba78-03dca3e7da62 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.354269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fad03f-6d3c-4811-ba78-03dca3e7da62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.368766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20898721-0cd2-4ee7-b146-df9e6828d6c7 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.376507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-20fad03f-6d3c-4811-ba78-03dca3e7da62 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:13.378438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-20fad03f-6d3c-4811-ba78-03dca3e7da62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:13.417504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:13.418554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.419076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.419577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.420507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:13.421013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:13.423121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:13.440301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 310/624] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:13.442139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.448970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 315/625] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:13.450084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fad03f-6d3c-4811-ba78-03dca3e7da62 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.498538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:13.572810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d24c58e-a7f2-4e1f-b761-2c44056d37c5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.575132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fad03f-6d3c-4811-ba78-03dca3e7da62 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.589964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:13.612633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:13.613784 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.614009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.614378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.619823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:13.620793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.621508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.621932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.622304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:13.624825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:13.630612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:13.632303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:13.641142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:13.642157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:13.643306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 316/626] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:00:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:13.644489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:13.651069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:13.652054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:13.653265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:13.653265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:13.653265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:13.653265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:13.653265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:13.653265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:13.653265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:13.653265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:13.657298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 311/627] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:13.658263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.760991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1f000a-88ca-473d-9027-3d24a5c619bd None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.782627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:13.787239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aeec77-5fcf-4871-b69a-13dec669f6a0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.812387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:13.813970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.814421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.815008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.816376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:13.817119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:13.819209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:13.827491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171bb80-4d0e-4115-bd63-5394280673be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:13.828616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171bb80-4d0e-4115-bd63-5394280673be None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:13.828928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171bb80-4d0e-4115-bd63-5394280673be None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:13.829315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7171bb80-4d0e-4115-bd63-5394280673be None admin] PATH_INFO: `/v3/roles/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:13.841241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7171bb80-4d0e-4115-bd63-5394280673be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:13.842664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 312/628] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:00:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:13.843820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-7171bb80-4d0e-4115-bd63-5394280673be None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:13.849792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:13.851021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:13.852127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:13.858695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 317/629] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:13.859720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171bb80-4d0e-4115-bd63-5394280673be None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:13.882930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:13.964897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7171bb80-4d0e-4115-bd63-5394280673be None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:13.972751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e81cbba-b787-47e0-b5c9-e60ef8427887 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:14.004592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e0a4-7c2a-4afa-a3d1-638ea4f530cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.005934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e0a4-7c2a-4afa-a3d1-638ea4f530cc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.006621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e0a4-7c2a-4afa-a3d1-638ea4f530cc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.007110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4461e0a4-7c2a-4afa-a3d1-638ea4f530cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.014237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.014953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.015362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.015723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.016611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:14.017031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.018962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.035569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 313/630] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:14.036826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.040785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:14.098930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461e0a4-7c2a-4afa-a3d1-638ea4f530cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.101328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4461e0a4-7c2a-4afa-a3d1-638ea4f530cc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.152436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:14.168062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32da2d4-d31e-40a4-84f0-39281cbe3213 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:14.204832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.205709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.206134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.206500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.207451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:14.208051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.209409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.223189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 314/631] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:14.224192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.226563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 318/632] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:14.228859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e0a4-7c2a-4afa-a3d1-638ea4f530cc None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.296560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:14.336536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4461e0a4-7c2a-4afa-a3d1-638ea4f530cc None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:14.338888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abc3f7c-6e8e-4136-bf9c-6962c9941cc0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:14.368885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.369685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.369967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.370243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.370632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb0a9c0-1e7e-47d8-8ceb-5437a3ab70aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.371546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a9c0-1e7e-47d8-8ceb-5437a3ab70aa None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.371825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a9c0-1e7e-47d8-8ceb-5437a3ab70aa None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.372100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb0a9c0-1e7e-47d8-8ceb-5437a3ab70aa None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/0c8203a8a7464c3bac93e5d42808c00f/roles/99ad2a72e54b4e51baabe132c647c8e3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.380002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.381466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.386420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0a9c0-1e7e-47d8-8ceb-5437a3ab70aa None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=0c8203a8a7464c3bac93e5d42808c00f, role_id=99ad2a72e54b4e51baabe132c647c8e3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.387881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb0a9c0-1e7e-47d8-8ceb-5437a3ab70aa None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:14.399078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:14.400200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:14.401277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:14.401277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:14.401277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:14.401277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:14.401277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 319/633] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:14.401277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.407957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 315/634] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:14 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/0c8203a8a7464c3bac93e5d42808c00f/roles/99ad2a72e54b4e51baabe132c647c8e3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:00:14.409302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76400a-7dc1-400e-8123-f2c42262ca38 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.409590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76400a-7dc1-400e-8123-f2c42262ca38 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.409865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef76400a-7dc1-400e-8123-f2c42262ca38 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.411122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 316/635] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:14.415813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a68d-1b8b-421c-9a85-228558e5d35e None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.415813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a68d-1b8b-421c-9a85-228558e5d35e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.415813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a68d-1b8b-421c-9a85-228558e5d35e None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.415813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 317/636] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:14.415813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a68d-1b8b-421c-9a85-228558e5d35e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.421788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:14.516611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f7951a-4573-4053-87ed-a17426cabe44 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:14.529141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:14.559045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6bea72-7c99-4d97-af12-828b23d68516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.559526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a68d-1b8b-421c-9a85-228558e5d35e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:14.559835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bea72-7c99-4d97-af12-828b23d68516 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.560081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bea72-7c99-4d97-af12-828b23d68516 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.560354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6bea72-7c99-4d97-af12-828b23d68516 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.573395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bea72-7c99-4d97-af12-828b23d68516 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.574891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6bea72-7c99-4d97-af12-828b23d68516 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-3b6bea72-7c99-4d97-af12-828b23d68516 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:14.594835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:14.596278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:14.597178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:14.597178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:14.597178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:14.597178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:14.597178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 320/637] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:14.597997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6aa8-a2dd-4f05-93fb-b25413b01a96 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.598285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6aa8-a2dd-4f05-93fb-b25413b01a96 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.598561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaf6aa8-a2dd-4f05-93fb-b25413b01a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.608668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2622fa22-b3da-43e4-8cc0-62a192e7da9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.608865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622fa22-b3da-43e4-8cc0-62a192e7da9f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.609098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622fa22-b3da-43e4-8cc0-62a192e7da9f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.609324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2622fa22-b3da-43e4-8cc0-62a192e7da9f None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.610056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622fa22-b3da-43e4-8cc0-62a192e7da9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:14.610383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622fa22-b3da-43e4-8cc0-62a192e7da9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.611776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2622fa22-b3da-43e4-8cc0-62a192e7da9f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.619820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-eaaf6aa8-a2dd-4f05-93fb-b25413b01a96 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:14.621777 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-eaaf6aa8-a2dd-4f05-93fb-b25413b01a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:14.623900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 318/638] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:14.625188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb63d1-e3cc-4aee-99c6-b0481645446e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.625462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb63d1-e3cc-4aee-99c6-b0481645446e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.625738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb63d1-e3cc-4aee-99c6-b0481645446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.642267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-86fb63d1-e3cc-4aee-99c6-b0481645446e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:14.643476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-86fb63d1-e3cc-4aee-99c6-b0481645446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:14.694035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 321/639] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:14.694940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf6aa8-a2dd-4f05-93fb-b25413b01a96 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.707198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:14.722033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 319/640] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:14.724145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb63d1-e3cc-4aee-99c6-b0481645446e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.812595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:14.825962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaf6aa8-a2dd-4f05-93fb-b25413b01a96 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:14.842649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb63d1-e3cc-4aee-99c6-b0481645446e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:14.862646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.863450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.863734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.864007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.864735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:14.865040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.866694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.876498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:14.877294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:14.877577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:14.877849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:14.883794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 322/641] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:14.883794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.889923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:14.891433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:14.905862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:14.906810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:14.907791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 320/642] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:14.908409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:14.971011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:14.992131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a92b17-567e-4150-81a3-c03afa1302f9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.019217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a73e30-222b-4988-b291-9ba9a63f3b5b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.027796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bffd0c-de8a-4b22-afb1-ae7b692c472e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.028607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bffd0c-de8a-4b22-afb1-ae7b692c472e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.028884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bffd0c-de8a-4b22-afb1-ae7b692c472e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.029309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bffd0c-de8a-4b22-afb1-ae7b692c472e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.040955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bffd0c-de8a-4b22-afb1-ae7b692c472e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-33bffd0c-de8a-4b22-afb1-ae7b692c472e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:15.048943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:15.049909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:15.050768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 323/643] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:15.051500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bffd0c-de8a-4b22-afb1-ae7b692c472e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.059851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc52598-db44-45ec-bc90-8594d134025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.060638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc52598-db44-45ec-bc90-8594d134025a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.060936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc52598-db44-45ec-bc90-8594d134025a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.061200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc52598-db44-45ec-bc90-8594d134025a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.067298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:15.072329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc52598-db44-45ec-bc90-8594d134025a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-6cc52598-db44-45ec-bc90-8594d134025a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:15.079972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:15.080894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:15.081791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 321/644] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:15.082598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc52598-db44-45ec-bc90-8594d134025a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.165404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bffd0c-de8a-4b22-afb1-ae7b692c472e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.190273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc52598-db44-45ec-bc90-8594d134025a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.201625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.202429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.202704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.202972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.205349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:15.210504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.212094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:15.231429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:15.232373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:15.233262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 324/645] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:15.233754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.237846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.238734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.239189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.239853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.241863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:15.242359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.244025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.267803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 322/646] 10.222.0.69 () {56 vars in 1137 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:15.268957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.308019 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:15.338349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705d702-1510-4d9a-ab8f-e716fe920379 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.372121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.373032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.373310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.373589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.374341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:15.374644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.376062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.377665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f7092-0a18-4946-9c14-8d6d3ee9123e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.383833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 325/647] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:15.384614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.415740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.416488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.416772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.417044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.417800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:15.418103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.419499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.432573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 323/648] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:15.432573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.462114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:15.492374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc32990d-c693-445f-92dc-b6a492d9f09b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.526224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.526909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.527192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.527498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.528304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:15.528612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.530107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.532042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:15.540796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a78f26f-d13e-4800-9aa9-87098460bbca None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.544283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 326/649] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:15.545276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.577500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740ab65-7411-49ed-a42e-d92f0f176c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.578173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740ab65-7411-49ed-a42e-d92f0f176c3f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.578453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740ab65-7411-49ed-a42e-d92f0f176c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.578724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740ab65-7411-49ed-a42e-d92f0f176c3f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.645961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740ab65-7411-49ed-a42e-d92f0f176c3f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.646979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e1aad8-050b-4dbf-b1c7-6f42a71eb025 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.647467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740ab65-7411-49ed-a42e-d92f0f176c3f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.676638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:15.682949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.683779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.684060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.684338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.695737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.697237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:15.714993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:15.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:15.716999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:15.716999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:15.716999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:15.716999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:15.716999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 327/650] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:15.716999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.753195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 324/651] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:15.754446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740ab65-7411-49ed-a42e-d92f0f176c3f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.772403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:15.833137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe9545-5f33-4b51-97da-f815eea83e9a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.860416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740ab65-7411-49ed-a42e-d92f0f176c3f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:15.868001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.868800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.869081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.869356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.879901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.881377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.893286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:15.898738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:15.899766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:15.900787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:15.900787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:15.900787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:15.900787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:15.900787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 328/652] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:15.901126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:15.905369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:15.906339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:15.906821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:15.907229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:15.908226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=04cd27bb637a404d827d384196eabfb1) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:15.908552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:15.909981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:15.916938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 325/653] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/role_assignments?user.id=04cd27bb637a404d827d384196eabfb1&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:15.917673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.039028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cfd0b-ed33-47a2-a56d-5e6ad83b36af None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.044661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:16.049808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923c276-c31c-4d6f-9bc5-30158e517256 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.087950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.088706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.089050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.089385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.090233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:16.090556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.092099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.092758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.094114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.094592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.095706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.110001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 329/654] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:16.110001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.115523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.117285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:16.139122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:16.140111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:16.140985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:16.140985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:16.140985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:16.140985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:16.143077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 326/655] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:16.144199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.211580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:16.233525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c4616-e906-4bb7-b3fa-caca569c94b9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.278067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.278763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.279047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.279375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.280130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:16.280500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.281986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.296047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 330/656] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:16.297013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.326600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:16.400935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1356a8a-740f-4809-933f-acd437807d7f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.409130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d79eb-313d-4146-9cc1-b89eb1ce4ea0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.437670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.439808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.440349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.441937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.450544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.451251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.451575 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.451965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.452606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:16.452921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.453752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.454428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.455661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.473611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 331/657] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:16.474898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:16.475453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:16.476389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 20 09:00:16.477758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:16.481307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 327/658] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:16.482233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.583724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:16.590895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8789a3de-90cd-4fec-883f-cd9c5a1fca7d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.601221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:16.618961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3168ef04-d679-43e0-84c7-9b676b587552 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.628973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.629855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.630556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.630556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.637902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.639513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:16.653393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:16.654291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:16.655153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 328/659] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:16.655977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.664674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c33b57-35b4-468e-989a-bcef1c9d09a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.665816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c33b57-35b4-468e-989a-bcef1c9d09a2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.666097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c33b57-35b4-468e-989a-bcef1c9d09a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.666363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c33b57-35b4-468e-989a-bcef1c9d09a2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.673761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c33b57-35b4-468e-989a-bcef1c9d09a2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.675209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c33b57-35b4-468e-989a-bcef1c9d09a2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-c6c33b57-35b4-468e-989a-bcef1c9d09a2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:16.689853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:16.690944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 20 09:00:16.692084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:16.695412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 332/660] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:16.696868 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb1c99-f5d3-481e-900e-52766e6c8c03 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.697159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb1c99-f5d3-481e-900e-52766e6c8c03 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.697507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb1c99-f5d3-481e-900e-52766e6c8c03 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.700826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 333/661] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:16.701774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb1c99-f5d3-481e-900e-52766e6c8c03 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.766241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d2c93-87e1-4921-a1a3-e1685ebd014e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.806555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.807249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.807536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.807857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.808646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:16.808955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.810397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.813639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb1c99-f5d3-481e-900e-52766e6c8c03 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.827936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 329/662] 10.222.0.69 () {56 vars in 1139 bytes} [Fri May 20 09:00:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:16.832967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:16.852047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6c1c97-0c25-4209-949f-5585fab1a765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.852711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c1c97-0c25-4209-949f-5585fab1a765 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.852991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c1c97-0c25-4209-949f-5585fab1a765 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.853265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6c1c97-0c25-4209-949f-5585fab1a765 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.854458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c1c97-0c25-4209-949f-5585fab1a765 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:16.854773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c1c97-0c25-4209-949f-5585fab1a765 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.856242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6c1c97-0c25-4209-949f-5585fab1a765 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:16.859730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:16.863517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:16.870675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 334/663] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:16.872007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f768-1bb6-48ee-8063-27c6d63edafe None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.872292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f768-1bb6-48ee-8063-27c6d63edafe None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.872583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f768-1bb6-48ee-8063-27c6d63edafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.943028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0057dd9-91a6-46f4-a2f8-25e3741b0384 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:16.990175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:16.990870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:16.991386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:16.991657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:16.992516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:16.992820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:16.994288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:17.019405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 330/664] 10.222.0.69 () {56 vars in 1137 bytes} [Fri May 20 09:00:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:17.020124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.025520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-1a00f768-1bb6-48ee-8063-27c6d63edafe None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:17.026700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1a00f768-1bb6-48ee-8063-27c6d63edafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:17.097740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 335/665] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:17.100121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00f768-1bb6-48ee-8063-27c6d63edafe None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.110654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:17.129472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f318d64-dc97-4db0-b870-54b36efe9c23 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:17.166795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172f26c-db1a-4d0e-8d4e-04ec5534bc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:17.167465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172f26c-db1a-4d0e-8d4e-04ec5534bc07 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:17.167741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172f26c-db1a-4d0e-8d4e-04ec5534bc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:17.168005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7172f26c-db1a-4d0e-8d4e-04ec5534bc07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:17.213506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00f768-1bb6-48ee-8063-27c6d63edafe None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:17.235210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172f26c-db1a-4d0e-8d4e-04ec5534bc07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:17.236718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7172f26c-db1a-4d0e-8d4e-04ec5534bc07 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:17.241231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:17.249942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:17.250605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:17.251166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:17.251166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:17.252025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=4b672a2898b849338844b1c8e911646d) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:17.252606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:17.253822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:17.271390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 336/666] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/role_assignments?user.id=4b672a2898b849338844b1c8e911646d&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:17.271390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.340375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:17.358771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 331/667] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:17.359864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172f26c-db1a-4d0e-8d4e-04ec5534bc07 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.418936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95dee95-a8d8-4774-aece-341ae1a8558d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:17.467550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2eced-9bff-4773-86e9-d1a6cbb9fc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:17.468201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7172f26c-db1a-4d0e-8d4e-04ec5534bc07 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:17.468319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2eced-9bff-4773-86e9-d1a6cbb9fc3d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:17.468504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2eced-9bff-4773-86e9-d1a6cbb9fc3d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:17.468781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2eced-9bff-4773-86e9-d1a6cbb9fc3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:17.513727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000a7da-6389-47b2-987b-9d9f35134dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:17.514848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000a7da-6389-47b2-987b-9d9f35134dfe None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:17.515309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000a7da-6389-47b2-987b-9d9f35134dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:17.515591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000a7da-6389-47b2-987b-9d9f35134dfe None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:17.532328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b000a7da-6389-47b2-987b-9d9f35134dfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b000a7da-6389-47b2-987b-9d9f35134dfe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:17.542127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:17.543090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 332/668] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:17.544840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000a7da-6389-47b2-987b-9d9f35134dfe None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.554424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:17.569317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2eced-9bff-4773-86e9-d1a6cbb9fc3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:17.570924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e2eced-9bff-4773-86e9-d1a6cbb9fc3d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:17.605298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:17.662413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b000a7da-6389-47b2-987b-9d9f35134dfe None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:17.678410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 337/669] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:17.678410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2eced-9bff-4773-86e9-d1a6cbb9fc3d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.703441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:17.704591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:17.705086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:17.705569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:17.717267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:17.719013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:17.736896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:17.737872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:17.738757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:17.738757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:17.738757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:17.738757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:17.741051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 333/670] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:17.742074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.773199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:17.782087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e2eced-9bff-4773-86e9-d1a6cbb9fc3d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:17.818316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:17.818908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:17.819185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:17.819617 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:17.820382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:17.820686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:17.822134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:17.830843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 338/671] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:17.831782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.855845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:17.862694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4222363-4afd-4d32-80ca-8498f2e34ec2 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:17.906199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:17.907343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:17.907828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:17.908268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:17.922880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:17.925145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:17.936237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d17804-abd4-4325-b774-ce20de9c66b7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:17.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:17.954077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:17.955060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:17.955060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:17.955060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:17.955060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:17.961665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 334/672] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:17.961665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:17.983151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:17.984268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:17.984765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:17.985231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:17.986381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:17.987122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:17.988691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.009150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 339/673] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:18.009150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.026393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:18.098387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bfb613-f5ee-49e8-8d49-370fcd03241c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.123618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd4f4c-f19e-47b9-8bff-12614b1bb75b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.145753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.147084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.147531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.147990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.159978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.160788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.161145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.161516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.162416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:18.162748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.163515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.164405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.165658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.176350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:18.181666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 340/674] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:18.182682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:18.190033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:18.191119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:18.192288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:18.192288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:18.192288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:18.192288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:18.194154 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 335/675] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:18.195802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.263658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:18.290354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfa9f9-e6d9-4455-872d-7ad45cefb0e0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.307688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8ab1a4-9f95-4737-8be4-5070db7eb5f4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.324973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.325770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.326048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.326321 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.333291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.334518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.341705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66a4142-40c4-4896-8dd0-4bb05c7045cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.342480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a4142-40c4-4896-8dd0-4bb05c7045cf None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.342743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a4142-40c4-4896-8dd0-4bb05c7045cf None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.343033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66a4142-40c4-4896-8dd0-4bb05c7045cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:18.348363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:18.349292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:18.350177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 341/676] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:18.350665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.355139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a4142-40c4-4896-8dd0-4bb05c7045cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.356616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66a4142-40c4-4896-8dd0-4bb05c7045cf None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-e66a4142-40c4-4896-8dd0-4bb05c7045cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:18.374666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:18.375662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:18.376534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 336/677] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:18.377403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07ae24-8f06-4810-8922-216b4ce2f492 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.377998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07ae24-8f06-4810-8922-216b4ce2f492 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.378340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07ae24-8f06-4810-8922-216b4ce2f492 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.379859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 337/678] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:18.380385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07ae24-8f06-4810-8922-216b4ce2f492 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.448135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:18.462700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc1e8f6-0774-4f0a-85d5-5b8f20b93967 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.495469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07ae24-8f06-4810-8922-216b4ce2f492 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.500418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab1b995-6c5f-436d-b9b4-3417ee31d62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.501092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b995-6c5f-436d-b9b4-3417ee31d62e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.501374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b995-6c5f-436d-b9b4-3417ee31d62e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.501645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab1b995-6c5f-436d-b9b4-3417ee31d62e None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.502375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1b995-6c5f-436d-b9b4-3417ee31d62e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:18.502675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1b995-6c5f-436d-b9b4-3417ee31d62e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.504488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab1b995-6c5f-436d-b9b4-3417ee31d62e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.506448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:18.523167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 342/679] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:18.524903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a743857-0dea-4403-9331-bfa2de7d6367 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.525185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a743857-0dea-4403-9331-bfa2de7d6367 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.525454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a743857-0dea-4403-9331-bfa2de7d6367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.530430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.531055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.532174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.532174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.532516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:18.532796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.534225 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.551303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 338/680] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:18.551570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0a743857-0dea-4403-9331-bfa2de7d6367 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:18.551761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.551941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0a743857-0dea-4403-9331-bfa2de7d6367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:18.627508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 343/681] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:18.627508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a743857-0dea-4403-9331-bfa2de7d6367 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.661722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed3197-f0e5-414f-a1e0-083eb56aed2a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.682090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:18.697694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.698353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.698626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.698903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.699702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:18.700005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.701789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.718712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 339/682] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:18.718712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.735690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a743857-0dea-4403-9331-bfa2de7d6367 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.762045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:18.784080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.784889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.785163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.785430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.794419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.795942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:18.810612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:18.811613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 344/683] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:18.812536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.830247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9c846-8e33-457f-827c-c00eef1735fe None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.865823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.866615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.866860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.867100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.867959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=349e99511c7c4307b27dc90b01f2dcd3) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:18.868310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:18.869690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:18.875645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 340/684] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/role_assignments?user.id=349e99511c7c4307b27dc90b01f2dcd3&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:18.884365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:18.920457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0ba9af-9b0b-4aa9-b68d-473ce61b3ee2 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:18.948732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:18.982391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14449175-2cda-427e-9924-3f270ccd8f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:18.984209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14449175-2cda-427e-9924-3f270ccd8f20 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:18.984624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14449175-2cda-427e-9924-3f270ccd8f20 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:18.984992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14449175-2cda-427e-9924-3f270ccd8f20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:18.997506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14449175-2cda-427e-9924-3f270ccd8f20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-14449175-2cda-427e-9924-3f270ccd8f20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:19.005716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:19.006920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:19.008021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:19.010083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 345/685] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:19.011118 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14449175-2cda-427e-9924-3f270ccd8f20 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:19.025420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd702bdb-400b-4e53-b90a-72eee8982ff7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.070620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:19.075106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:19.075940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.076355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.076664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.077691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=68d25a45f9ee4ae58c6751a8781d12d1) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:19.078014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:19.079641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:19.091396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 341/686] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:18 2022] GET /identity/v3/role_assignments?user.id=68d25a45f9ee4ae58c6751a8781d12d1&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:19.092462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:19.138563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14449175-2cda-427e-9924-3f270ccd8f20 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.178257 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:19.179081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.179391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.179671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.180639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:19.180955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:19.182678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:19.203990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 346/687] 10.222.0.69 () {56 vars in 1135 bytes} [Fri May 20 09:00:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:19.209915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:19.235772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184501d-7e97-440a-891e-305e5cd47398 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.247388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:19.279836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:19.280686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.281066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.281439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.282501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:19.282706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:19.284274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:19.294972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 342/688] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:19.308429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:19.358485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702947-2993-4d8c-9ed5-a96b1b177a2e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.360495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:19.399581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3915d0d5-f73a-4fa9-b2b8-677b97287f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:19.400341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d0d5-f73a-4fa9-b2b8-677b97287f5d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.400717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d0d5-f73a-4fa9-b2b8-677b97287f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.401085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3915d0d5-f73a-4fa9-b2b8-677b97287f5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.439753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbc8a6-99d3-404f-92ce-51cb6c4f9adf None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.483970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:19.485218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.486040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.486386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.488364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915d0d5-f73a-4fa9-b2b8-677b97287f5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:19.490484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3915d0d5-f73a-4fa9-b2b8-677b97287f5d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:19.499480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:19.501161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:19.522396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:19.523630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:19.524537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:19.524537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:19.524537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:19.524537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:19.531418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 343/689] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:19.531785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:19.603637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:19.623368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 347/690] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:19.624740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318115d-b067-4e85-af11-7bad6f716814 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.625035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318115d-b067-4e85-af11-7bad6f716814 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.625314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e318115d-b067-4e85-af11-7bad6f716814 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.626827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 348/691] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:19.631547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318115d-b067-4e85-af11-7bad6f716814 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:19.636125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:19.670075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ecbce-cafc-4f44-a57c-3c7e84675aa0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.709644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f4db-cf41-4b16-953a-311e8f75d4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:19.710356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f4db-cf41-4b16-953a-311e8f75d4ec None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.710714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f4db-cf41-4b16-953a-311e8f75d4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.710926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f4db-cf41-4b16-953a-311e8f75d4ec None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.711662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f4db-cf41-4b16-953a-311e8f75d4ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:19.711966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f4db-cf41-4b16-953a-311e8f75d4ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:19.713336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56f4db-cf41-4b16-953a-311e8f75d4ec None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:19.726440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 344/692] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:19.732440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6422c-2062-4319-870f-ab342778d5f9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.732712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6422c-2062-4319-870f-ab342778d5f9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.732989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6422c-2062-4319-870f-ab342778d5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.750180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-69c6422c-2062-4319-870f-ab342778d5f9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:19.751395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-69c6422c-2062-4319-870f-ab342778d5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:19.753028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e318115d-b067-4e85-af11-7bad6f716814 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.795455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:19.796650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.797131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.797604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:19.808570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:19.810324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:19.822617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 345/693] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:19.823757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6422c-2062-4319-870f-ab342778d5f9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:19.829175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:19.830053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:19.830930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:19.830930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:19.830930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:19.830930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:19.833123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 349/694] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:19.843634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:19.846500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:19.915827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:19.946415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6422c-2062-4319-870f-ab342778d5f9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.977030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642d6cd-690a-45b7-92be-11206d433edf None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:19.988082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:19.989189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:19.989696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:19.990104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.000923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.002561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.015685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.016473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.016875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:20.017252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.018096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:20.018505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:20.019623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:20.020705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:20.021643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:20.022202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.028877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 346/695] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:00:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:20.030382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.034036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 350/696] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:20.038588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.110834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:20.144580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125658f1-9696-4236-8dec-90bfcc9c61b1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.171734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db311a5-ffb4-41cd-8e64-77e863a4ea0f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.181273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.182385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.182887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:20.183433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.184282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.185879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.188024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:20.203209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:20.204321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:20.205147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:20.213045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 347/697] 10.222.0.69 () {56 vars in 1129 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:20.213045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.216895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.217817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.218314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:20.218778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.219863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:20.220398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.222082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.242513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 351/698] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:20.243841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.319708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdb2fe-e271-4007-8186-3f2c702d0503 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.345109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea77e27-348b-4a6e-9689-fe5e75d9c863 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.354305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.355105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.355419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:20.355697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.358687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:20.363583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.365217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:20.380484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:20.381439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:20.382448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:20.382448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:20.382448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:20.382448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:20.382448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:20.382448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:20.382448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:20.382448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:20.384543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.385206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.385415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 348/699] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:20.385513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:20.385742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.386547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:20.386877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.391485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.393109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.400103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 352/700] 10.222.0.69 () {56 vars in 1124 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:20.401139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.463724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:20.499553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e61b4-88d7-4af9-bcaf-f658b6d65acc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.511614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a84e6-8940-46fb-98ed-c414cc707cb7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.534264 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0a4c2-281b-4132-95e7-ea55ef785903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.535291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a4c2-281b-4132-95e7-ea55ef785903 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.535717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a4c2-281b-4132-95e7-ea55ef785903 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:20.536130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf0a4c2-281b-4132-95e7-ea55ef785903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.547614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.548389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.548759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:20.549130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.550110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:20.550523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.552139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.571759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 353/701] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:20.572883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.602635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:20.609641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0a4c2-281b-4132-95e7-ea55ef785903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.611276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf0a4c2-281b-4132-95e7-ea55ef785903 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.680773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e4abf5-5aef-40a1-8011-9bba83d59099 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.703768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:20.717844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 349/702] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 20 09:00:20.718685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.719163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0a4c2-281b-4132-95e7-ea55ef785903 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.719581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.720009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:20.720303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:20.721196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:20.721598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:20.724642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:20.731740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 354/703] 10.222.0.69 () {56 vars in 1124 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:20.733839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:20.908550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:20.920620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf0a4c2-281b-4132-95e7-ea55ef785903 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.947402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27683f2-758a-4dc8-951a-319973060731 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:20.997604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:20.998931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:20.999734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.000241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.001159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:21.001922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.002291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.002652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.003615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=0c8203a8a7464c3bac93e5d42808c00f) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:21.004017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:21.005637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:21.013440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:21.014607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 350/704] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/role_assignments?user.id=0c8203a8a7464c3bac93e5d42808c00f&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:21.015395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:21.035207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:21.036416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:21.037533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:21.037533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:21.037533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:21.037533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:21.039615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 355/705] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:21.045783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:21.166950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffac1c1-a7e8-48c6-84fc-4595e984256c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:21.211074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052d1d0-19d0-42a6-9627-2a4ac7f2fc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:21.211961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d1d0-19d0-42a6-9627-2a4ac7f2fc5c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.212341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d1d0-19d0-42a6-9627-2a4ac7f2fc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.212747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052d1d0-19d0-42a6-9627-2a4ac7f2fc5c None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.213509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052d1d0-19d0-42a6-9627-2a4ac7f2fc5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:21.213813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052d1d0-19d0-42a6-9627-2a4ac7f2fc5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:21.215366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052d1d0-19d0-42a6-9627-2a4ac7f2fc5c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:21.235224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 351/706] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:21.240811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:21.268015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:21.360204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad0084f-83ba-4ed4-8114-2288fa4bec06 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:21.385352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0c6b9-6f72-402c-a5f0-d1d126010acd None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.385685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0c6b9-6f72-402c-a5f0-d1d126010acd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.385910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde0c6b9-6f72-402c-a5f0-d1d126010acd None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.388830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 352/707] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:21.392781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747d63b-b2b5-4083-a723-56a8173c9792 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.393065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747d63b-b2b5-4083-a723-56a8173c9792 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.393341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b747d63b-b2b5-4083-a723-56a8173c9792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.397256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:21.398564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.399173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.399570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.406801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:21.408375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:21.411533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b747d63b-b2b5-4083-a723-56a8173c9792 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:21.412942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b747d63b-b2b5-4083-a723-56a8173c9792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:21.424253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:21.425174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:21.426490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 356/708] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:21.431610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:21.432344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:21.503563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 353/709] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:21.527803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:21.531998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747d63b-b2b5-4083-a723-56a8173c9792 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:21.563706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49592911-7c7d-48dd-a596-d522a0abd98b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:21.600429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:21.601287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.601639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.601984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.602855 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:21.603231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:21.605054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:21.618345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 357/710] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:21.624088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:21.651136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b747d63b-b2b5-4083-a723-56a8173c9792 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:21.687837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:21.688974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.689444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.689886 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.690713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:21.692302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:21.704918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:21.705946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:21.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:21.711374 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 354/711] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:21 2022] GET /identity/v3/services/network => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:21.715893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:21.732750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:21.737062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8975d259-9a90-421f-b9ae-7ce22f064c76 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:21.768743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:21.777409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82f0f3-c72d-49fb-93ef-fd3f118e9c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:21.778154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f0f3-c72d-49fb-93ef-fd3f118e9c63 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.778394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f0f3-c72d-49fb-93ef-fd3f118e9c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.778685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82f0f3-c72d-49fb-93ef-fd3f118e9c63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.779575 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82f0f3-c72d-49fb-93ef-fd3f118e9c63 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=569782752acb4085ad742330af999bc2, user.id=04cd27bb637a404d827d384196eabfb1) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:21.779896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82f0f3-c72d-49fb-93ef-fd3f118e9c63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:21.781383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b82f0f3-c72d-49fb-93ef-fd3f118e9c63 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:21.791432 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 358/712] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:21 2022] GET /identity/v3/role_assignments?user.id=04cd27bb637a404d827d384196eabfb1&scope.project.id=569782752acb4085ad742330af999bc2&role.id=6dca675ade814980a252f800fb4fa3ad => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:21.793922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bddda-bb67-472e-a009-1f6c9b751e8c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.794612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bddda-bb67-472e-a009-1f6c9b751e8c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.796549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bddda-bb67-472e-a009-1f6c9b751e8c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.799002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 359/713] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:21.805805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9b277-029f-40cc-b0ad-dc48113636c9 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.806391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9b277-029f-40cc-b0ad-dc48113636c9 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.806838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9b277-029f-40cc-b0ad-dc48113636c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.838235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-acf9b277-029f-40cc-b0ad-dc48113636c9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:21.840460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-acf9b277-029f-40cc-b0ad-dc48113636c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:21.874680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5dfe55-b307-42d7-80b3-91200aa086b8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:21.916019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:21.917714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:21.918176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:21.918618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:21.919589 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] RBAC: Adding query filter params (name=network) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:21.920077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:21.921719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:21.933122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 355/714] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:21.934824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:22.003423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 360/715] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:22.048604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9b277-029f-40cc-b0ad-dc48113636c9 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:22.080623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:22.127113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2dd5b0-2db2-4c6a-a90c-bd01596f168e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:22.187241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9b277-029f-40cc-b0ad-dc48113636c9 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:22.188834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:22.189774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:22.190256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:22.190737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:22.191690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:22.192248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:22.193852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:22.210169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 356/716] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:22.231653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:22.237605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:22.238598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:22.239591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:22.239591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:22.251892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:22.253470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:22.273513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:22.274521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:22.275499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:22.275499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:22.275499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:22.275499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:22.279407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 361/717] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:22.281370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:22.421684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11114062-6da5-4e5e-a9eb-d4b13981cbd3 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:22.457137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e16cd5-1ed9-4eef-b7f6-80fdd7d60f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:22.458166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e16cd5-1ed9-4eef-b7f6-80fdd7d60f8f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:22.458649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e16cd5-1ed9-4eef-b7f6-80fdd7d60f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:22.459185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e16cd5-1ed9-4eef-b7f6-80fdd7d60f8f None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:22.460347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e16cd5-1ed9-4eef-b7f6-80fdd7d60f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:22.460887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e16cd5-1ed9-4eef-b7f6-80fdd7d60f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:22.462471 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e16cd5-1ed9-4eef-b7f6-80fdd7d60f8f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:22.476095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 357/718] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:22.481384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:22.586333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746c5d1f-9fc5-480d-8043-57793c70ae85 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:22.629947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57ab8b-af6f-4345-959a-fd5602d3148e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:22.630994 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ab8b-af6f-4345-959a-fd5602d3148e None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:22.631530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ab8b-af6f-4345-959a-fd5602d3148e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:22.632050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ab8b-af6f-4345-959a-fd5602d3148e None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:22.633004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57ab8b-af6f-4345-959a-fd5602d3148e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:22.634696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57ab8b-af6f-4345-959a-fd5602d3148e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:22.671622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 362/719] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 09:00:22 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:00:23.401308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59937663-dfc9-41eb-a045-382f8a3a4297 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:23.402329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59937663-dfc9-41eb-a045-382f8a3a4297 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:23.402839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59937663-dfc9-41eb-a045-382f8a3a4297 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:23.404733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 358/720] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:23.410670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd716d-2315-40f6-9526-062f7ab4a822 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:23.411084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd716d-2315-40f6-9526-062f7ab4a822 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:23.411292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd716d-2315-40f6-9526-062f7ab4a822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:23.428797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-7cfd716d-2315-40f6-9526-062f7ab4a822 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:23.430000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7cfd716d-2315-40f6-9526-062f7ab4a822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:23.456892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:23.515947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 363/721] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:23.544419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59937663-dfc9-41eb-a045-382f8a3a4297 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:23.551498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:23.666561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59937663-dfc9-41eb-a045-382f8a3a4297 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:23.705712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99726992-624f-4656-a8f5-c45c9eac56f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:23.706772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726992-624f-4656-a8f5-c45c9eac56f7 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:23.707281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726992-624f-4656-a8f5-c45c9eac56f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:23.707822 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99726992-624f-4656-a8f5-c45c9eac56f7 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:23.708786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726992-624f-4656-a8f5-c45c9eac56f7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:23.710439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99726992-624f-4656-a8f5-c45c9eac56f7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:23.732027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 359/722] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:00:23 2022] POST /identity/v3/services => generated 251 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:24.180905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed11345-3e9c-4791-8196-e71f15577523 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:24.181329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed11345-3e9c-4791-8196-e71f15577523 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:24.181505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed11345-3e9c-4791-8196-e71f15577523 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:24.183132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 364/723] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:24.189091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85501aaa-885c-4e7b-8a76-26aad015c51f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:24.189383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85501aaa-885c-4e7b-8a76-26aad015c51f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:24.189662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85501aaa-885c-4e7b-8a76-26aad015c51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:24.207506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-85501aaa-885c-4e7b-8a76-26aad015c51f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:24.208712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-85501aaa-885c-4e7b-8a76-26aad015c51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:24.281080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:24.307136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 360/724] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:24.329485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed11345-3e9c-4791-8196-e71f15577523 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:24.331475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6585a-beb3-4436-8e30-694dc2d49a96 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:24.331762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6585a-beb3-4436-8e30-694dc2d49a96 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:24.332032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6585a-beb3-4436-8e30-694dc2d49a96 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:24.334074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 361/725] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:24.339363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce84c7-5fbb-4d7c-ab98-7dea2b234a02 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:24.339644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce84c7-5fbb-4d7c-ab98-7dea2b234a02 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:24.339915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce84c7-5fbb-4d7c-ab98-7dea2b234a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:24.358975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-abce84c7-5fbb-4d7c-ab98-7dea2b234a02 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:24.360186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-abce84c7-5fbb-4d7c-ab98-7dea2b234a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:24.472728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 362/726] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:24 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 20 09:00:24.505224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed11345-3e9c-4791-8196-e71f15577523 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:24.508783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce84c7-5fbb-4d7c-ab98-7dea2b234a02 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:24.545461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135964c-57fd-4ea1-9270-5f781ed2041d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:24.546567 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135964c-57fd-4ea1-9270-5f781ed2041d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:24.546997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135964c-57fd-4ea1-9270-5f781ed2041d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:24.547773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6135964c-57fd-4ea1-9270-5f781ed2041d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:24.562565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6135964c-57fd-4ea1-9270-5f781ed2041d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-6135964c-57fd-4ea1-9270-5f781ed2041d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:24.571612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:24.572782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:24.573639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:24.576884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 365/727] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:24.584311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135964c-57fd-4ea1-9270-5f781ed2041d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:24.616846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:24.711605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6135964c-57fd-4ea1-9270-5f781ed2041d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:24.732244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce84c7-5fbb-4d7c-ab98-7dea2b234a02 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:24.754350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:24.773567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052960-ee54-493e-a235-dcda6288b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:24.774422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052960-ee54-493e-a235-dcda6288b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:24.774697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052960-ee54-493e-a235-dcda6288b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:24.774972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052960-ee54-493e-a235-dcda6288b1a7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:24.788235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:24.788908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:24.789189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:24.789460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:24.790555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:24.790860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:24.792268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:24.797327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58052960-ee54-493e-a235-dcda6288b1a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-58052960-ee54-493e-a235-dcda6288b1a7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:24.806208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:24.807141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:24.808021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:24.814790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 363/729] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:24.815182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 366/729] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:24.815857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052960-ee54-493e-a235-dcda6288b1a7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:24.819628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:24.964381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:24.981063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f2b17-4dc1-48e7-b719-9898ab4bc3f5 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.026613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58052960-ee54-493e-a235-dcda6288b1a7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.048261 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.049459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.049947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.050422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.061685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.063535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:25.073494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0b28ea-4062-47ff-9bf4-ac5cdf333da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.074169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b28ea-4062-47ff-9bf4-ac5cdf333da6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.074453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b28ea-4062-47ff-9bf4-ac5cdf333da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.074979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0b28ea-4062-47ff-9bf4-ac5cdf333da6 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.075778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b28ea-4062-47ff-9bf4-ac5cdf333da6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:25.076080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b28ea-4062-47ff-9bf4-ac5cdf333da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.077457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0b28ea-4062-47ff-9bf4-ac5cdf333da6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:25.083796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:25.084686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:25.085615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:25.085615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:25.085615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:25.085615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:25.086887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 364/730] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:25.088087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 367/731] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:25.088354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332f45f-7fb6-4024-a198-7baae1bf5a40 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.088641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332f45f-7fb6-4024-a198-7baae1bf5a40 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.088913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332f45f-7fb6-4024-a198-7baae1bf5a40 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.090117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 365/732] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:25.092358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:25.096365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b1171-2cdc-46ee-9a8f-f8e9721569b0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.096645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b1171-2cdc-46ee-9a8f-f8e9721569b0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.096921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b1171-2cdc-46ee-9a8f-f8e9721569b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.119213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-605b1171-2cdc-46ee-9a8f-f8e9721569b0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:25.121578 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-605b1171-2cdc-46ee-9a8f-f8e9721569b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:25.165986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:25.208094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedbfa3-3f55-4128-a959-1837e2846d56 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.208724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 366/733] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 20 09:00:25.209819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b1171-2cdc-46ee-9a8f-f8e9721569b0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:25.266293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.267109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.267421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.267699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.276254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:25.281722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.283047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:25.302431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:25.303404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:25.305050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:25.305050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:25.305050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:25.305050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:25.305050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 368/734] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:25.305050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:25.316781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b1171-2cdc-46ee-9a8f-f8e9721569b0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.352706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c624434-7907-45d4-a042-351681b208ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.353742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c624434-7907-45d4-a042-351681b208ab None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.354214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c624434-7907-45d4-a042-351681b208ab None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.354676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c624434-7907-45d4-a042-351681b208ab None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.355636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c624434-7907-45d4-a042-351681b208ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:25.356145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c624434-7907-45d4-a042-351681b208ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.357704 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c624434-7907-45d4-a042-351681b208ab None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:25.369814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 367/735] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:25.370983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c624434-7907-45d4-a042-351681b208ab None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:25.402130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:25.410401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dadd5ad-1035-4b2f-8425-3730d7768bfe None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.449913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e0d4a-82cd-4411-a1b6-8ff96c5fc160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.450590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e0d4a-82cd-4411-a1b6-8ff96c5fc160 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.450874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e0d4a-82cd-4411-a1b6-8ff96c5fc160 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.451153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e0d4a-82cd-4411-a1b6-8ff96c5fc160 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.451972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e0d4a-82cd-4411-a1b6-8ff96c5fc160 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.453518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e0d4a-82cd-4411-a1b6-8ff96c5fc160 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:25.479412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 369/736] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:00:25 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:25.481198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e0d4a-82cd-4411-a1b6-8ff96c5fc160 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:25.492450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c624434-7907-45d4-a042-351681b208ab None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.569860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:25.583216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.584027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.584309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.584580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.585486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:25.585790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.588682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:25.618438 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 368/737] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:25.626119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:25.675670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e0d4a-82cd-4411-a1b6-8ff96c5fc160 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.713061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041bf11e-ea2a-4dc5-b997-94663786b248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.714133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bf11e-ea2a-4dc5-b997-94663786b248 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.714659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bf11e-ea2a-4dc5-b997-94663786b248 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.715070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041bf11e-ea2a-4dc5-b997-94663786b248 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.723608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041bf11e-ea2a-4dc5-b997-94663786b248 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.725392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041bf11e-ea2a-4dc5-b997-94663786b248 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-041bf11e-ea2a-4dc5-b997-94663786b248 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:25.740599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:25.741693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:25.742610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:25.743775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:25.745089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 370/738] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:25.746176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d4981-eccd-463f-8ce3-07c94d80fd06 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.746428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d4981-eccd-463f-8ce3-07c94d80fd06 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.746689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d4981-eccd-463f-8ce3-07c94d80fd06 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.748516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 371/739] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:25.754071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437a332-65dc-4f05-a445-e1341114308b None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.754363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437a332-65dc-4f05-a445-e1341114308b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.754644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437a332-65dc-4f05-a445-e1341114308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.765139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d995e-90f4-46c9-9645-d5c9450f0547 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.775516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4437a332-65dc-4f05-a445-e1341114308b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:25.776736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4437a332-65dc-4f05-a445-e1341114308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:25.810142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.811302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.811759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.812222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.822199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.823984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:25.838187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:25.839145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:25.840167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:25.842786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 369/740] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:00:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:25.848219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:25.863614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 372/741] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:25.864916 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437a332-65dc-4f05-a445-e1341114308b None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:25.920243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:25.958416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1308f-2824-4b5f-8054-3d20181a9db2 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.994219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:25.995172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:25.995490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:25.995767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:25.996626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:25.996936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:25.997977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437a332-65dc-4f05-a445-e1341114308b None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:25.998452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.020566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 370/742] 10.222.0.69 () {56 vars in 1135 bytes} [Fri May 20 09:00:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:26.021417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.026762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:26.042899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.044406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.045104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.045569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.047213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:26.047763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:26.049421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.072574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 373/743] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:00:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:26.074092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.148159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd804d94-90c0-482a-90c3-a8f4e3c9ef02 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:26.190992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333e91a-1735-4540-b955-ed1dc0e041cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.192498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333e91a-1735-4540-b955-ed1dc0e041cc None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.192925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333e91a-1735-4540-b955-ed1dc0e041cc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.193391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1333e91a-1735-4540-b955-ed1dc0e041cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.200726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:26.206619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2cc95-9d42-4e08-a7ab-fc07e3d245c5 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:26.207759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1333e91a-1735-4540-b955-ed1dc0e041cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-1333e91a-1735-4540-b955-ed1dc0e041cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:26.217701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:26.218855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:26.219775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:26.222828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 371/744] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:26.223953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333e91a-1735-4540-b955-ed1dc0e041cc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.252207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff683a-3ea6-4097-b71d-ab70b5187f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.252896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff683a-3ea6-4097-b71d-ab70b5187f30 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.253176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff683a-3ea6-4097-b71d-ab70b5187f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.253493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff683a-3ea6-4097-b71d-ab70b5187f30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.303066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:26.346189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff683a-3ea6-4097-b71d-ab70b5187f30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:26.348281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaff683a-3ea6-4097-b71d-ab70b5187f30 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.354095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1333e91a-1735-4540-b955-ed1dc0e041cc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:26.401612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.402450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.402726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.403002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.416850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:26.418762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:26.437570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:26.438529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:26.439459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:26.439459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:26.439459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:26.439459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:26.446139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 372/745] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:26.447300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.465669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:26.472327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 374/746] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:00:26.476488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff683a-3ea6-4097-b71d-ab70b5187f30 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.566790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b9151-cf27-4ff3-ba7d-b98b4bffce88 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:26.584834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:26.604167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.604774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.605057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.605326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.606063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:26.606365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:26.607769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.613866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff683a-3ea6-4097-b71d-ab70b5187f30 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:26.616340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 373/747] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:26.617113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.656298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccb99c1-99a0-4c32-b905-bb4d47dd273c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.657281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb99c1-99a0-4c32-b905-bb4d47dd273c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.657830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb99c1-99a0-4c32-b905-bb4d47dd273c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.658352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccb99c1-99a0-4c32-b905-bb4d47dd273c None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.659727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb99c1-99a0-4c32-b905-bb4d47dd273c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:26.660299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb99c1-99a0-4c32-b905-bb4d47dd273c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:26.661869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccb99c1-99a0-4c32-b905-bb4d47dd273c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.674543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 375/748] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:26.675833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea996-ec0b-4535-b63d-796f97393745 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.676119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea996-ec0b-4535-b63d-796f97393745 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.676522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ea996-ec0b-4535-b63d-796f97393745 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.681338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 376/749] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:26.681338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ea996-ec0b-4535-b63d-796f97393745 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.729124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29145311-8b5c-4c25-8980-f519209c946d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:26.739149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:26.767394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.767394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.767394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.767394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.779576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:26.781505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.792039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ea996-ec0b-4535-b63d-796f97393745 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:26.801314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:26.802455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:26.803999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:26.803999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:26.803999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:26.803999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:26.807230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 374/750] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:26.808111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.838717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f9d2a3-085c-4255-b22a-9c815f02cbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.839818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d2a3-085c-4255-b22a-9c815f02cbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.840098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d2a3-085c-4255-b22a-9c815f02cbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.840508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9d2a3-085c-4255-b22a-9c815f02cbf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.840805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:26.854631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9d2a3-085c-4255-b22a-9c815f02cbf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:26.856456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f9d2a3-085c-4255-b22a-9c815f02cbf0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-12f9d2a3-085c-4255-b22a-9c815f02cbf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:26.873982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:26.874876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:26.876179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:26.876179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:26.876179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:26.876179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:26.876179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 377/751] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:26.876797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41bae7-7b91-4262-8736-57358a775af0 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.877082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41bae7-7b91-4262-8736-57358a775af0 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.877352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41bae7-7b91-4262-8736-57358a775af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.894116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0e41bae7-7b91-4262-8736-57358a775af0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:26.895327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0e41bae7-7b91-4262-8736-57358a775af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:26.915864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed5de5-53a4-4f22-a2be-e9a20d15828f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:26.955843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:26.956640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:26.956919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:26.957194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:26.963904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:26.964776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 378/752] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:26.965350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:26.965979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41bae7-7b91-4262-8736-57358a775af0 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:26.990219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:26.991246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:26.992182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 375/753] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:26.992664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.021117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:27.081304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41bae7-7b91-4262-8736-57358a775af0 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.085743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:27.097255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3c88d-1f21-4ec0-bd26-709c58ab131e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.117318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.118310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.118602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.118883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.119659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:27.119971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.121369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.132473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.135305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 379/754] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:27.135305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.137707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.138001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.138280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.139020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:27.139351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.140734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.153180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 376/755] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:27.158108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.232987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:27.236227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd9157-80c6-4da7-8f91-aee7d141debf None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.264126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e05feea-16be-4834-af00-76ac21613a53 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.270936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.271850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.272703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.273035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.273769 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.275864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:27.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:27.291171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:27.292346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 380/756] 10.222.0.69 () {56 vars in 1129 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:27.293396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.306997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.307694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.307972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.308244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.309034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:27.309342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.310758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.328829 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 377/757] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:27.329709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.341934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:27.404120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d04cb-c63e-4903-a2ed-4cb0844e8199 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.439001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556e21c1-a021-4f68-9455-7df4d6d44004 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.440825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.441704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.441991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.442265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.450343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.451835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:27.466045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:27.466942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 381/758] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:27.468155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.477830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.478639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.478915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.479160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.487121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.488696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.498338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:27.505988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:27.506983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:27.507956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 378/759] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:27.508559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.583901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:27.588792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d480028-c146-4cbc-8e34-a60c95fc7ebc None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.620511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b947bd54-41c4-450d-9b2e-773863069976 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.625613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.626287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.626570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.626847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.627588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:27.627897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.629690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.639646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 382/760] 10.222.0.69 () {56 vars in 1124 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:27.639646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.668199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe220e-107f-497b-9d92-6f82f1133b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.669103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe220e-107f-497b-9d92-6f82f1133b30 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.669289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe220e-107f-497b-9d92-6f82f1133b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.669562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe220e-107f-497b-9d92-6f82f1133b30 None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/04cd27bb637a404d827d384196eabfb1/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.671842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe220e-107f-497b-9d92-6f82f1133b30 None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=04cd27bb637a404d827d384196eabfb1, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.673312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe220e-107f-497b-9d92-6f82f1133b30 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.690675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 379/761] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:27 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/04cd27bb637a404d827d384196eabfb1/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:00:27.691545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe220e-107f-497b-9d92-6f82f1133b30 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.746967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:27.769565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c251aa-23c1-4964-b85b-66f070e25449 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.817028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.818275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.818717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.819171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.820930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:27.821462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.823839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.842723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 383/762] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:27.843939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.844054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe220e-107f-497b-9d92-6f82f1133b30 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.877949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:27.885041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3985766-1736-43df-b360-a1fbffb8bdc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.886017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3985766-1736-43df-b360-a1fbffb8bdc6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.886539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3985766-1736-43df-b360-a1fbffb8bdc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.887026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3985766-1736-43df-b360-a1fbffb8bdc6 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.888112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3985766-1736-43df-b360-a1fbffb8bdc6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:27.889071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3985766-1736-43df-b360-a1fbffb8bdc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.891109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3985766-1736-43df-b360-a1fbffb8bdc6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.911354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 380/763] 10.222.0.69 () {56 vars in 1149 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:27.912954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf9d41-84ac-402e-8c01-910a8cf6f53e None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.913437 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf9d41-84ac-402e-8c01-910a8cf6f53e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.913881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cf9d41-84ac-402e-8c01-910a8cf6f53e None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.915969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 381/764] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:27.916846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf9d41-84ac-402e-8c01-910a8cf6f53e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:27.952425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd6f8b-f1e5-4769-bcec-5ae4b6d74e4a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:27.985344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:27.986014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:27.986292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:27.986561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:27.987287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:27.987604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:27.988979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:27.998329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 384/765] 10.222.0.69 () {56 vars in 1124 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:27.998329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:28.010247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:28.035817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cf9d41-84ac-402e-8c01-910a8cf6f53e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:28.071003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f8d20-7607-41e6-a127-97b02c37c4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:28.071687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8d20-7607-41e6-a127-97b02c37c4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:28.071970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8d20-7607-41e6-a127-97b02c37c4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:28.072247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8d20-7607-41e6-a127-97b02c37c4d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:28.073084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f8d20-7607-41e6-a127-97b02c37c4d4 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=569782752acb4085ad742330af999bc2, user.id=0c8203a8a7464c3bac93e5d42808c00f) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:28.073397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f8d20-7607-41e6-a127-97b02c37c4d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:28.074830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f8d20-7607-41e6-a127-97b02c37c4d4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:28.087286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 382/766] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/role_assignments?user.id=0c8203a8a7464c3bac93e5d42808c00f&scope.project.id=569782752acb4085ad742330af999bc2&role.id=6dca675ade814980a252f800fb4fa3ad => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:28.087286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc669-0529-4fee-8cda-1f36c991ceae None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:28.087286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc669-0529-4fee-8cda-1f36c991ceae None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:28.087286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffcc669-0529-4fee-8cda-1f36c991ceae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:28.110193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496fcafe-aa98-446a-bfe3-9d22a19cbc17 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:28.112218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-fffcc669-0529-4fee-8cda-1f36c991ceae None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:28.114475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-fffcc669-0529-4fee-8cda-1f36c991ceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:28.152519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79344b-c245-4ba0-92b6-261712222a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:28.153476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79344b-c245-4ba0-92b6-261712222a56 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:28.153764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79344b-c245-4ba0-92b6-261712222a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:28.154045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f79344b-c245-4ba0-92b6-261712222a56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:28.176130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:28.201748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 383/767] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:28.202894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcc669-0529-4fee-8cda-1f36c991ceae None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:28.249563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79344b-c245-4ba0-92b6-261712222a56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:28.252033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f79344b-c245-4ba0-92b6-261712222a56 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:28.270689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:28.325774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffcc669-0529-4fee-8cda-1f36c991ceae None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:28.360333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:28.360992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:28.361273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:28.361541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:28.362337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] RBAC: Adding query filter params (interface=public, service_id=d6909af97d7d4da6821dbe290d85bff0, region_id=RegionOne) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:28.362637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:28.364229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:28.372198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 384/768] 10.222.0.69 () {56 vars in 1251 bytes} [Fri May 20 09:00:28 2022] GET /identity/v3/endpoints?service_id=d6909af97d7d4da6821dbe290d85bff0&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:28.373110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:28.383687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 385/769] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:00:28.385272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79344b-c245-4ba0-92b6-261712222a56 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:28.426549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:28.489044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2ccc1-03b7-4a60-a98f-ec65fca1e3c5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:28.513289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f79344b-c245-4ba0-92b6-261712222a56 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:28.529860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:28.536886 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:28.539125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:28.540251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:28.540703 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] PATH_INFO: `/v3/services/network` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:28.541783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:28.544056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:28.553056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f34b-01e5-47aa-a393-2bd163b12b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:28.553901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f34b-01e5-47aa-a393-2bd163b12b95 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:28.554298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f34b-01e5-47aa-a393-2bd163b12b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:28.554640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774f34b-01e5-47aa-a393-2bd163b12b95 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:28.555443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774f34b-01e5-47aa-a393-2bd163b12b95 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:28.556885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774f34b-01e5-47aa-a393-2bd163b12b95 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:28.560813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:28.561773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:28.562629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:28.563410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 386/770] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:00:28 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:28.563485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f34b-01e5-47aa-a393-2bd163b12b95 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:28.565713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 385/771] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:28 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:28.575352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:28.699420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774f34b-01e5-47aa-a393-2bd163b12b95 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:28.736887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:28.738004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:28.738500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:28.738908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:28.750338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:28.752136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:28.770910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:28.772126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:28.773107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:28.773107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:28.773107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:28.773107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:28.774752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 387/772] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:28.780639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:28.816715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45db002f-e96c-45c5-b54e-b25d984c6f6e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:28.830089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:28.847409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:28.872870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:28.873772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:28.874147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:28.874518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:28.875495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] RBAC: Adding query filter params (name=network) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:28.875898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:28.877975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:28.889881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 386/773] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:28.891058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:28.953336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0d6da7-d3e9-454d-b920-d41f2d6ca951 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:28.999542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:29.000837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:29.001333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:29.001821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:29.002881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:29.003495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:29.005088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:29.031419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 388/774] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:29.032784 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:29.033431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679db01-fdbd-404c-a0fa-9d0d99403587 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:29.074325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:29.074992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:29.076349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:29.076633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:29.077413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] RBAC: Adding query filter params (type=network) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:29.077711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:29.081204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:29.093504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 387/775] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:29.096212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:29.176774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:29.230875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:29.235798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57a2db-8880-43d7-877f-161400a02ebc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:29.262944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d614b28e-d92f-484d-a7b6-47a52006c9d5 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:29.283548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:29.284213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:29.284492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:29.284761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:29.285560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] RBAC: Adding query filter params (interface=public, service_id=92df69621acc4d548a4dd9a1e1459956, region_id=RegionOne) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:29.285864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:29.287247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:29.301102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 388/776] 10.222.0.69 () {56 vars in 1251 bytes} [Fri May 20 09:00:29 2022] GET /identity/v3/endpoints?service_id=92df69621acc4d548a4dd9a1e1459956&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:29.306683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:29.335000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:29.335859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:29.336137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:29.336407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:29.352439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:29.353929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:29.385348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:29.386308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:29.387202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 389/777] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:29.393460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:29.511362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5098468-6f79-4025-a87e-131c898c6ad4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:29.519656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:29.572474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76bf1-c19a-45a6-929e-7e3233dbe36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:29.573150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76bf1-c19a-45a6-929e-7e3233dbe36c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:29.573492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76bf1-c19a-45a6-929e-7e3233dbe36c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:29.573772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f76bf1-c19a-45a6-929e-7e3233dbe36c None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:29.574505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f76bf1-c19a-45a6-929e-7e3233dbe36c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:29.575967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f76bf1-c19a-45a6-929e-7e3233dbe36c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:29.586841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 389/778] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:00:29 2022] GET /identity/v3/services => generated 772 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:29.600966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7219345-da81-408e-8f06-ab125bf8c2e9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:29.644149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b165af-67aa-4a17-8d26-2652332ab36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:29.645248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b165af-67aa-4a17-8d26-2652332ab36b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:29.645736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b165af-67aa-4a17-8d26-2652332ab36b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:29.646212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b165af-67aa-4a17-8d26-2652332ab36b None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:29.647514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b165af-67aa-4a17-8d26-2652332ab36b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:29.648047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b165af-67aa-4a17-8d26-2652332ab36b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:29.649751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b165af-67aa-4a17-8d26-2652332ab36b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:29.673865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 390/779] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:29.679950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76bf1-c19a-45a6-929e-7e3233dbe36c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:29.825297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f76bf1-c19a-45a6-929e-7e3233dbe36c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:29.865297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b4407-5d9d-4587-887a-0a40b5ae097d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:29.865958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b4407-5d9d-4587-887a-0a40b5ae097d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:29.866267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b4407-5d9d-4587-887a-0a40b5ae097d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:29.866544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b4407-5d9d-4587-887a-0a40b5ae097d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:29.867417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b4407-5d9d-4587-887a-0a40b5ae097d None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=6cd71c35c4224658aa2ab0b81f0f4b97) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:29.867729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b4407-5d9d-4587-887a-0a40b5ae097d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:29.869171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46b4407-5d9d-4587-887a-0a40b5ae097d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:29.877591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 390/780] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:29 2022] GET /identity/v3/role_assignments?user.id=6cd71c35c4224658aa2ab0b81f0f4b97&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:30.186224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b32f7-434e-45dd-8753-b0e5507e0f1a None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:30.187217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b32f7-434e-45dd-8753-b0e5507e0f1a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:30.187729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b32f7-434e-45dd-8753-b0e5507e0f1a None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:30.189333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 391/781] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:30.195105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837e23-1aae-4894-8776-b838d8b5f9a1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:30.195424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837e23-1aae-4894-8776-b838d8b5f9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:30.195700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837e23-1aae-4894-8776-b838d8b5f9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:30.216182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f5837e23-1aae-4894-8776-b838d8b5f9a1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:30.217349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f5837e23-1aae-4894-8776-b838d8b5f9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:30.256454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:30.310362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 391/782] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:30.352875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b32f7-434e-45dd-8753-b0e5507e0f1a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:30.373775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:30.497016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b32f7-434e-45dd-8753-b0e5507e0f1a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:30.543588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafbdb5-d14d-4566-a2e5-d061a0f0bc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:30.544406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafbdb5-d14d-4566-a2e5-d061a0f0bc45 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:30.544688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafbdb5-d14d-4566-a2e5-d061a0f0bc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:30.544958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafbdb5-d14d-4566-a2e5-d061a0f0bc45 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:30.555989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafbdb5-d14d-4566-a2e5-d061a0f0bc45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-2dafbdb5-d14d-4566-a2e5-d061a0f0bc45 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:30.563602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:30.564537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:30.565367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 392/783] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:30.571001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5837e23-1aae-4894-8776-b838d8b5f9a1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:30.580080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcb66c-a3df-4bed-a624-ddb832247bd7 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:30.580368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcb66c-a3df-4bed-a624-ddb832247bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:30.580646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fcb66c-a3df-4bed-a624-ddb832247bd7 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:30.581994 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 393/784] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:30.587988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b21d07-c477-417e-8b25-d5434f1370c2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:30.588276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b21d07-c477-417e-8b25-d5434f1370c2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:30.588990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b21d07-c477-417e-8b25-d5434f1370c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:30.608147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c8b21d07-c477-417e-8b25-d5434f1370c2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:30.609358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c8b21d07-c477-417e-8b25-d5434f1370c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:30.688882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:30.689440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 394/785] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:30.713830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b21d07-c477-417e-8b25-d5434f1370c2 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:30.789226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5837e23-1aae-4894-8776-b838d8b5f9a1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:30.837813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:30.838985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:30.839287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:30.839590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:30.840492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:30.840825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:30.842546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:30.862904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 392/786] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:30.866854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:30.917344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:30.937066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b21d07-c477-417e-8b25-d5434f1370c2 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.016108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a25282-6923-4a40-ae7d-8e10ef0448d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.016946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a25282-6923-4a40-ae7d-8e10ef0448d6 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.017230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a25282-6923-4a40-ae7d-8e10ef0448d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.017509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a25282-6923-4a40-ae7d-8e10ef0448d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.036861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a25282-6923-4a40-ae7d-8e10ef0448d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:31.037619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f30e9-e707-4955-ab4d-5558d159d2e1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-10a25282-6923-4a40-ae7d-8e10ef0448d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:31.053175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:31.054422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:31.055449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 395/787] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:31.060979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a25282-6923-4a40-ae7d-8e10ef0448d6 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.084767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.087429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.087855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.088289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.120312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.121948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:31.147579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:31.155358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:31.156575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:31.157692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:31.157692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:31.157692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:31.157692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:31.158663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 393/788] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:31.163996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.196819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a25282-6923-4a40-ae7d-8e10ef0448d6 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.236314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539bbe76-9c7a-491f-bb63-32f8330baa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.237402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bbe76-9c7a-491f-bb63-32f8330baa8b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.237858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bbe76-9c7a-491f-bb63-32f8330baa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.238630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539bbe76-9c7a-491f-bb63-32f8330baa8b None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.239877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bbe76-9c7a-491f-bb63-32f8330baa8b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:31.240483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bbe76-9c7a-491f-bb63-32f8330baa8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.242671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539bbe76-9c7a-491f-bb63-32f8330baa8b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:31.254220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 396/789] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:31.255827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d2926-c85d-4cd3-86cf-ce0168363606 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.256248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d2926-c85d-4cd3-86cf-ce0168363606 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.256700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41d2926-c85d-4cd3-86cf-ce0168363606 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.258401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 397/790] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:31.259560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d2926-c85d-4cd3-86cf-ce0168363606 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.298370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:31.330676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357adbf-f9f1-4612-aacc-542c0840012b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.366580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24eb5ba-ead0-4931-817e-250f55700d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.367364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eb5ba-ead0-4931-817e-250f55700d19 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.367731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eb5ba-ead0-4931-817e-250f55700d19 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.368069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24eb5ba-ead0-4931-817e-250f55700d19 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.368947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24eb5ba-ead0-4931-817e-250f55700d19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:31.369308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24eb5ba-ead0-4931-817e-250f55700d19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.370836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24eb5ba-ead0-4931-817e-250f55700d19 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:31.376561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41d2926-c85d-4cd3-86cf-ce0168363606 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.387909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 394/791] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:31.387909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50de62-a0d0-42c2-9c31-8a01d05044e7 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.387909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50de62-a0d0-42c2-9c31-8a01d05044e7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.387909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb50de62-a0d0-42c2-9c31-8a01d05044e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.408057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:31.411847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-eb50de62-a0d0-42c2-9c31-8a01d05044e7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:31.413638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-eb50de62-a0d0-42c2-9c31-8a01d05044e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:31.425897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.426785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.427136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.427500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.440665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.442699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:31.464979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:31.465941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:31.466829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:31.466829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:31.466829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:31.466829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:31.466829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 398/792] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:31.468172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.500240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 395/793] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:31.501412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50de62-a0d0-42c2-9c31-8a01d05044e7 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.564196 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:31.586088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bfb23f-9a9e-4165-921e-41db0635d0d7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.604306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb50de62-a0d0-42c2-9c31-8a01d05044e7 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.621957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.623185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.623751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.624235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.632515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.634244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:31.640417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.641133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.641479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.641759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.642536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:31.642839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.644263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:31.650543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:31.651537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:00:31.652486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:31.659364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 396/795] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:31.659364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.659611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 399/795] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:31.660810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.692674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:31.762614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3e9571-f26e-454f-9b18-2b9a6fc4ee10 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.769477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6042c61c-82c1-4bd6-a184-7ac3b9d3f90b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.795513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:31.808578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.809026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.809264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.809549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.809826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.810680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:31.810787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.810987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.811492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.811953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.812476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:31.813022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.814950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:31.832691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:31.833965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 20 09:00:31.834854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:31.836804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 400/796] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:31.837706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.838523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 397/797] 10.222.0.69 () {56 vars in 1129 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:31.839975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:31.933806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:31.944775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f8efe-75f6-4c45-a239-a309505a5c97 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.975605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ce2fb7-0c98-4198-9780-3f4454fc65c5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:31.982815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:31.983644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:31.983928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:31.984501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:31.991579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:31.993043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:32.007606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:32.008560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 401/798] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:32.010101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.017289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168178a-63b4-4ca5-b7e8-b33c8ee095b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.018283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168178a-63b4-4ca5-b7e8-b33c8ee095b3 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.018756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168178a-63b4-4ca5-b7e8-b33c8ee095b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.019219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1168178a-63b4-4ca5-b7e8-b33c8ee095b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.076528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:32.091062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168178a-63b4-4ca5-b7e8-b33c8ee095b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.092842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1168178a-63b4-4ca5-b7e8-b33c8ee095b3 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.112727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294331b1-1db7-415e-9ee7-02e2dfca2832 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:32.155903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.156603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.156892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.157161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.157909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:32.158218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.159692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.165678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 402/799] 10.222.0.69 () {56 vars in 1124 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:32.166674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.183990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:32.208109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 398/800] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:31 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:00:32.209363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168178a-63b4-4ca5-b7e8-b33c8ee095b3 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.278293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b04ed6-eda9-49e1-baef-ec75b448c69e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:32.300673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:32.314518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1168178a-63b4-4ca5-b7e8-b33c8ee095b3 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:32.314702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.315196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.315510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.315778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.316696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:32.317003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.318471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.335953 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 403/801] 10.222.0.69 () {56 vars in 1135 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:32.336694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.349390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bce32a-81e0-45f4-b03f-16cd90103114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.350050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bce32a-81e0-45f4-b03f-16cd90103114 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.350326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bce32a-81e0-45f4-b03f-16cd90103114 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.350647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bce32a-81e0-45f4-b03f-16cd90103114 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.351390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bce32a-81e0-45f4-b03f-16cd90103114 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:32.351693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bce32a-81e0-45f4-b03f-16cd90103114 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.353055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bce32a-81e0-45f4-b03f-16cd90103114 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.357850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 399/802] 10.222.0.69 () {56 vars in 1124 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:32.359082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e967b-c054-470a-922e-d11a8d0fe9cf None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.359391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e967b-c054-470a-922e-d11a8d0fe9cf None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.359664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e967b-c054-470a-922e-d11a8d0fe9cf None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.363404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 400/803] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:32.363404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1648473-1358-46e2-8efd-cc1432669478 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.363404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1648473-1358-46e2-8efd-cc1432669478 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.363404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1648473-1358-46e2-8efd-cc1432669478 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.367275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 401/804] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:32.367275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1648473-1358-46e2-8efd-cc1432669478 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.409499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:32.439240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc5288-1212-4b7d-a882-d372e00e1ed1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:32.468564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1648473-1358-46e2-8efd-cc1432669478 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:32.474460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.475283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.475644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.475917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.485976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.487509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.501869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be7d780-d8d8-40ae-83a5-c4c4aa09803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.502661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7d780-d8d8-40ae-83a5-c4c4aa09803f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.502945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7d780-d8d8-40ae-83a5-c4c4aa09803f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.503216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be7d780-d8d8-40ae-83a5-c4c4aa09803f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:32.505240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:32.506082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:32.507111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:32.507111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:32.507111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:32.507111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:32.507111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 404/805] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:32.507417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.513667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7d780-d8d8-40ae-83a5-c4c4aa09803f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.515178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be7d780-d8d8-40ae-83a5-c4c4aa09803f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-6be7d780-d8d8-40ae-83a5-c4c4aa09803f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:32.533381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:32.534263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:32.535131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:32.535131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:32.535131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:32.535131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:32.535131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 402/806] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:32.536078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a5a5-6794-416a-9aed-51f5e10f3469 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.536353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a5a5-6794-416a-9aed-51f5e10f3469 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.536630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8966a5a5-6794-416a-9aed-51f5e10f3469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.542641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:32.553263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8966a5a5-6794-416a-9aed-51f5e10f3469 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:32.554480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8966a5a5-6794-416a-9aed-51f5e10f3469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:32.627950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba9bca-0fb8-4414-9fcc-16597cb9a16a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:32.628608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 403/807] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:32.629900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c6ea-35dc-4203-b79c-9200ccea7f7d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.630190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c6ea-35dc-4203-b79c-9200ccea7f7d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.630454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50c6ea-35dc-4203-b79c-9200ccea7f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.655882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:32.656594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-db50c6ea-35dc-4203-b79c-9200ccea7f7d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:32.657859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-db50c6ea-35dc-4203-b79c-9200ccea7f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:32.671331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fcdf5-0e8c-43de-a3d6-cc9ba02ca83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.672038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fcdf5-0e8c-43de-a3d6-cc9ba02ca83d None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.672325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fcdf5-0e8c-43de-a3d6-cc9ba02ca83d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.672603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05fcdf5-0e8c-43de-a3d6-cc9ba02ca83d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.673369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fcdf5-0e8c-43de-a3d6-cc9ba02ca83d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.674878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05fcdf5-0e8c-43de-a3d6-cc9ba02ca83d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.698030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 405/808] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:00:32 2022] POST /identity/v3/endpoints => generated 346 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:32.698865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fcdf5-0e8c-43de-a3d6-cc9ba02ca83d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.753366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 404/809] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:32.754166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50c6ea-35dc-4203-b79c-9200ccea7f7d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.837813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05fcdf5-0e8c-43de-a3d6-cc9ba02ca83d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:32.838900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:32.893775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50c6ea-35dc-4203-b79c-9200ccea7f7d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:32.901560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.902669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.902848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.903097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.904463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:32.904463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.905592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.921522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 406/810] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:32.922571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.943885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:32.944612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:32.946272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:32.946272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:32.946272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:32.946272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:32.947610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:32.962449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 405/811] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:32.967308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:32.979729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:33.047342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed42f9b7-62e7-44ad-9d40-8f22f1b80c4e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.079820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a499d-46d0-4895-8866-c43215d48b85 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.084804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.086890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.086890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.086890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] PATH_INFO: `/v3/services/network` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.087782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.089599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:33.103024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:33.104227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:33.105077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 407/812] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/services/network => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:33.106372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.110429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:33.128724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542e8d2-a722-4451-8120-e5cca525f1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.129538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e8d2-a722-4451-8120-e5cca525f1af None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.129816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e8d2-a722-4451-8120-e5cca525f1af None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.130092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7542e8d2-a722-4451-8120-e5cca525f1af None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.142026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7542e8d2-a722-4451-8120-e5cca525f1af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-7542e8d2-a722-4451-8120-e5cca525f1af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:33.152628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:33.153612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 406/813] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:33.155090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542e8d2-a722-4451-8120-e5cca525f1af None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.237134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a929b7-e1cb-40a9-bb5b-6f2bfaabaa88 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.240060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:33.270898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7542e8d2-a722-4451-8120-e5cca525f1af None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.279833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.280645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.280930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.281253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.288877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.290677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:33.305020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:33.305997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:33.306893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 408/814] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:33.308276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.315794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.316592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.316871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.317152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.324796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.326331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:33.342183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:33.343165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:33.344126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:33.344126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:33.344126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:33.344126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:33.344126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:33.344126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:33.344126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:33.344126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:33.347390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 407/815] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:33.349462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.396343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:33.428940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcc73c8-7908-4f13-a1dd-c00a9bab430b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.455905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca3965d-70da-48af-9bd3-0baafb7dfdd0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.466433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.467114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.467426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.467703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.468426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] RBAC: Adding query filter params (name=network) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:33.468733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.470111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.475045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 409/816] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:33.476141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.490832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.491596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.491877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.492150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.492908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:33.493211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.494610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.507894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 408/817] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:33.508796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.509253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:33.583918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9314f9-17a8-4152-b110-eb2ec1a68457 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.613007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32ef9c4-784c-4b20-87b5-3daa94bf4493 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.614880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:33.625761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.626436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.626724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.627062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.627907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:33.628223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.629681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.641285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 410/818] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:33.642106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.658011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.658888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.659448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.659883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.660916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:33.661370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.663102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.675252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 409/819] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:33.676060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.739306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:33.752508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4311038-f73e-4178-9b99-73ed7a30d446 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.781992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5f263-fad3-45ac-94bc-48b9222526b1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.796284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.797189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.797484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.797763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.798490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:33.798801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.800224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.805110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 411/820] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:33.805891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.823642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.824718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.825201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.825644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.836946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.838394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:33.855319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:33.861426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:33.862553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:33.862553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:33.862553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:33.862553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:33.862553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 410/821] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:33.862553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.878175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:33.919244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1c094c-5c8c-44dd-a5c1-6bb49ae900a1 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.954929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:33.955618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:33.955904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:33.956182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:33.957025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=569782752acb4085ad742330af999bc2, user.id=04cd27bb637a404d827d384196eabfb1) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:33.957336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:33.958786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:33.965512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 412/822] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:33 2022] GET /identity/v3/role_assignments?user.id=04cd27bb637a404d827d384196eabfb1&scope.project.id=569782752acb4085ad742330af999bc2&role.id=6dca675ade814980a252f800fb4fa3ad => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:33.966283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:33.981145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92721029-b3cd-40a5-be61-c6f0481aa5d7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:33.994686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:34.024716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7df4c-3d94-475e-9389-a18a1d009dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:34.025621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7df4c-3d94-475e-9389-a18a1d009dd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:34.025898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7df4c-3d94-475e-9389-a18a1d009dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:34.026169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7df4c-3d94-475e-9389-a18a1d009dd1 None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/0c8203a8a7464c3bac93e5d42808c00f/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:34.028278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7df4c-3d94-475e-9389-a18a1d009dd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=0c8203a8a7464c3bac93e5d42808c00f, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:34.029721 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7df4c-3d94-475e-9389-a18a1d009dd1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:34.055297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 411/823] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:33 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/0c8203a8a7464c3bac93e5d42808c00f/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:00:34.055297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7df4c-3d94-475e-9389-a18a1d009dd1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:34.158937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:34.192636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2d012-c482-4b71-98e2-fd66d797114a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:34.196845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7df4c-3d94-475e-9389-a18a1d009dd1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:34.240387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20c586-6a4a-4bed-a7d5-400bb80bef2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:34.240607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20c586-6a4a-4bed-a7d5-400bb80bef2b None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:34.240922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20c586-6a4a-4bed-a7d5-400bb80bef2b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:34.241403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20c586-6a4a-4bed-a7d5-400bb80bef2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:34.242268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20c586-6a4a-4bed-a7d5-400bb80bef2b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:34.245495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:34.246485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:34.246678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b20c586-6a4a-4bed-a7d5-400bb80bef2b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:34.247143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:34.247645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:34.248587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:34.249074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:34.251061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:34.268895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 412/824] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:34.274160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:34.282463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 413/825] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:00:33 2022] POST /identity/v3/endpoints => generated 327 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:34.391512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971d4fbf-8f0a-4c6a-ae56-dfd913ef2b37 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:34.432946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:34.440716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:34.441607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:34.441962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:34.442241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:34.450871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:34.452439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:34.470105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:34.471115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:34.472094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 413/826] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:00:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:34.477218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20c586-6a4a-4bed-a7d5-400bb80bef2b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:34.563637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:34.598533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20c586-6a4a-4bed-a7d5-400bb80bef2b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:34.635614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:34.636303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:34.636588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:34.636863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:34.637718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:34.638032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:34.639518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:34.662178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 414/827] 10.222.0.69 () {56 vars in 1149 bytes} [Fri May 20 09:00:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:34.664159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:34.774795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de8179-450d-46af-ae5f-2e306c37e58b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:34.810945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98824fc-9029-4a8b-bea2-ea991c03c0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:34.812134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98824fc-9029-4a8b-bea2-ea991c03c0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:34.812613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98824fc-9029-4a8b-bea2-ea991c03c0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:34.813100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98824fc-9029-4a8b-bea2-ea991c03c0d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:34.824612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98824fc-9029-4a8b-bea2-ea991c03c0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:34.826406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98824fc-9029-4a8b-bea2-ea991c03c0d7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-c98824fc-9029-4a8b-bea2-ea991c03c0d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:34.847989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:34.848957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:34.849832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:34.849832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:34.849832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:34.849832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:34.852440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 414/828] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:34.859629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:34.981699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9f3326-abe4-46e5-b2ea-3a50e77f5375 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:35.011704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:35.018039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73772315-191c-4d7e-8db8-8cda9b4e6323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:35.018722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772315-191c-4d7e-8db8-8cda9b4e6323 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.019009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772315-191c-4d7e-8db8-8cda9b4e6323 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.019348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73772315-191c-4d7e-8db8-8cda9b4e6323 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.020092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73772315-191c-4d7e-8db8-8cda9b4e6323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:35.021298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73772315-191c-4d7e-8db8-8cda9b4e6323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:35.022776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73772315-191c-4d7e-8db8-8cda9b4e6323 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:35.035574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701236b-f69a-4d46-9949-9f4522b87394 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.036101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701236b-f69a-4d46-9949-9f4522b87394 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.036101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7701236b-f69a-4d46-9949-9f4522b87394 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.037700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 415/830] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:35.037869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 415/830] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:35.042844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701236b-f69a-4d46-9949-9f4522b87394 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:35.043603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55195750-f12a-4e61-9b12-1b00cf6f1e37 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.043901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55195750-f12a-4e61-9b12-1b00cf6f1e37 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.044187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55195750-f12a-4e61-9b12-1b00cf6f1e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.065285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-55195750-f12a-4e61-9b12-1b00cf6f1e37 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:35.066766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-55195750-f12a-4e61-9b12-1b00cf6f1e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:35.099501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:35.146020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 416/831] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:35 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:35.163934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55195750-f12a-4e61-9b12-1b00cf6f1e37 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:35.168174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7701236b-f69a-4d46-9949-9f4522b87394 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:35.212213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:35.213269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.213555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.213838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.223249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:35.225598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:35.244311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:35.245399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:35.246563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:35.246563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:35.246563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:35.246563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:35.246563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:35.246563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:35.246563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:35.246563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:35.250600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 416/832] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:35.251936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:35.274934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55195750-f12a-4e61-9b12-1b00cf6f1e37 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:35.296244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:35.314925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:35.316560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.316997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.317472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.318514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:35.320386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:35.336494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:35.337694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:35.338761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:35.341554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 417/833] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:35.348067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:35.378708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc75052a-ae56-402b-bfbc-3a13d5f2ffa9 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:35.380194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:35.417872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:35.418930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.419434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.419887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.420859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:35.421393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:35.423407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:35.442701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 417/834] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:35.443929 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:35.460788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea695530-1114-4cc9-9883-8f799ea24a6e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:35.496856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:35.497683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.498311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.498660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.499587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:35.500068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:35.501783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:35.511561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 418/835] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:35.512954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:35.563511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:35.564125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e473d52-c28f-4d61-a7c7-c5be731aff36 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:35.602315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2e6df-281d-4e98-b54a-6dda7fc1ab51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:35.603710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2e6df-281d-4e98-b54a-6dda7fc1ab51 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.604188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2e6df-281d-4e98-b54a-6dda7fc1ab51 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.604658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2e6df-281d-4e98-b54a-6dda7fc1ab51 None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/6cd71c35c4224658aa2ab0b81f0f4b97/roles/99ad2a72e54b4e51baabe132c647c8e3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.615110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3423e23-f3c1-4d7c-9371-45fa8f2238eb None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:35.621024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2e6df-281d-4e98-b54a-6dda7fc1ab51 None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=6cd71c35c4224658aa2ab0b81f0f4b97, role_id=99ad2a72e54b4e51baabe132c647c8e3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:35.622846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2e6df-281d-4e98-b54a-6dda7fc1ab51 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:35.647648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 418/836] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:35 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/6cd71c35c4224658aa2ab0b81f0f4b97/roles/99ad2a72e54b4e51baabe132c647c8e3 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:00:35.652803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf33af-0155-425a-b78d-c768fc9940ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:35.653546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf33af-0155-425a-b78d-c768fc9940ee None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:35.653834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf33af-0155-425a-b78d-c768fc9940ee None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:35.654160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf33af-0155-425a-b78d-c768fc9940ee None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:35.654999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf33af-0155-425a-b78d-c768fc9940ee None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:35.655341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf33af-0155-425a-b78d-c768fc9940ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:35.657007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf33af-0155-425a-b78d-c768fc9940ee None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:35.663377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 419/837] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:35 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:36.509646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9af9fa-cd38-4091-b2ca-6af5cc8c4681 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:36.510201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9af9fa-cd38-4091-b2ca-6af5cc8c4681 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:36.510201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9af9fa-cd38-4091-b2ca-6af5cc8c4681 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:36.511857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 419/838] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:36.517251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59c997-8279-41a3-9782-69bdc13f7f1f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:36.518002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59c997-8279-41a3-9782-69bdc13f7f1f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:36.518419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f59c997-8279-41a3-9782-69bdc13f7f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:36.529698 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67042d43-bbc3-41f9-9dd5-3a0c18d23f92 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:36.530029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67042d43-bbc3-41f9-9dd5-3a0c18d23f92 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:36.530258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67042d43-bbc3-41f9-9dd5-3a0c18d23f92 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:36.531794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 420/839] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:36.536870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50342298-ffa7-4fb7-9d7e-f68425e7f009 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:36.537155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50342298-ffa7-4fb7-9d7e-f68425e7f009 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:36.537434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50342298-ffa7-4fb7-9d7e-f68425e7f009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:36.537861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3f59c997-8279-41a3-9782-69bdc13f7f1f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:36.539708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3f59c997-8279-41a3-9782-69bdc13f7f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:36.557485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-50342298-ffa7-4fb7-9d7e-f68425e7f009 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:36.558950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-50342298-ffa7-4fb7-9d7e-f68425e7f009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:36.568543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:36.610926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 420/840] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:36.635156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59c997-8279-41a3-9782-69bdc13f7f1f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:36.637759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 421/841] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:36.659768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50342298-ffa7-4fb7-9d7e-f68425e7f009 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:36.683594 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:36.741433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f59c997-8279-41a3-9782-69bdc13f7f1f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:36.761732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50342298-ffa7-4fb7-9d7e-f68425e7f009 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:36.786982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593ecc9-1b1d-4f81-a538-b465eaad3214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:36.788298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593ecc9-1b1d-4f81-a538-b465eaad3214 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:36.788901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593ecc9-1b1d-4f81-a538-b465eaad3214 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:36.789726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593ecc9-1b1d-4f81-a538-b465eaad3214 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:36.796732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:36.797628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:36.797823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:36.798098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:36.798761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:36.800176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:36.802373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4593ecc9-1b1d-4f81-a538-b465eaad3214 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-4593ecc9-1b1d-4f81-a538-b465eaad3214 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:36.810752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:36.811844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:00:36.812688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:36.813427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:36.815004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:36.815891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:36.823586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 422/843] 10.222.0.69 () {56 vars in 1132 bytes} [Fri May 20 09:00:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:36.823586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:36.823983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 421/843] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:36.825318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593ecc9-1b1d-4f81-a538-b465eaad3214 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:36.832690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:36.928836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a3a57c-6dd3-4a71-b4ff-ddde8d5fdf02 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:36.934892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4593ecc9-1b1d-4f81-a538-b465eaad3214 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:36.963209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:36.963550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:36.964184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:36.964465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:36.964738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:36.965446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:36.965748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:36.967124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:36.975925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:36.976638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 423/844] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:36 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:36.977095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:36.977514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:36.977848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:36.978087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:36.979181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:36.979713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:36.981368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:36.995531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 422/845] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:36.997155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:37.048588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:37.079782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5047d5-4e38-48c5-bf44-38cdadc74ed0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:37.101939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79247004-b88c-4b55-8fa0-594af9e64b62 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:37.114439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:37.115506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:37.115948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:37.116326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:37.117288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:37.117695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:37.119845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:37.127325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 424/846] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:36 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:37.145497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:37.146597 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:37.147017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:37.147509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:37.159594 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:37.161438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:37.181401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:37.182534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:37.183678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:37.183678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:37.183678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:37.183678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:37.185650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 423/847] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:37.191834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:37.302986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3855e6c-8c4c-4145-851d-692867f18f22 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:37.340648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:37.341654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:37.342085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:37.342359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:37.343086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:37.343422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:37.344803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:37.346340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:37.358917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 425/848] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:37.364412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:37.383851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:37.467703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301350c1-f47d-4a06-b28d-b34fd4517d51 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:37.503087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:37.504000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:37.504321 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:37.504634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:37.513194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:37.514673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:37.529239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:37.530636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:37.531620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 424/849] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:00:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:37.536488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:37.643680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029a0ae-267b-43dc-94fa-8f0f1c0a15b7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:37.680519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279878c-cae6-4cb8-835b-bfcdd9e2eb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:37.681741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279878c-cae6-4cb8-835b-bfcdd9e2eb3d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:37.682222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279878c-cae6-4cb8-835b-bfcdd9e2eb3d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:37.682694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279878c-cae6-4cb8-835b-bfcdd9e2eb3d None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:37.683829 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279878c-cae6-4cb8-835b-bfcdd9e2eb3d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:37.684333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279878c-cae6-4cb8-835b-bfcdd9e2eb3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:37.685999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279878c-cae6-4cb8-835b-bfcdd9e2eb3d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:37.710284 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 426/850] 10.222.0.69 () {56 vars in 1135 bytes} [Fri May 20 09:00:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:37.711799 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:37.818315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:37.842010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269586f-258a-466d-856b-f24cdccc55d2 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:37.876156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0f219-98d3-459d-8604-05bfb255eeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:37.876810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0f219-98d3-459d-8604-05bfb255eeb7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:37.877091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0f219-98d3-459d-8604-05bfb255eeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:37.877366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0f219-98d3-459d-8604-05bfb255eeb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:37.933180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b96b0-2291-44fc-be55-12cb5ad820fa None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:37.933773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b96b0-2291-44fc-be55-12cb5ad820fa None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:37.934248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b96b0-2291-44fc-be55-12cb5ad820fa None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:37.936167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 427/851] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:37.941880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6cbef-362b-490e-a72f-1bc8e6178f10 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:37.942484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6cbef-362b-490e-a72f-1bc8e6178f10 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:37.942965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6cbef-362b-490e-a72f-1bc8e6178f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:37.947858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0f219-98d3-459d-8604-05bfb255eeb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:37.949347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0f219-98d3-459d-8604-05bfb255eeb7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:37.963348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-66e6cbef-362b-490e-a72f-1bc8e6178f10 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:37.964773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-66e6cbef-362b-490e-a72f-1bc8e6178f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:38.013986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:38.030722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:38.036960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 428/852] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:38.038692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baf39f-5483-46a9-9cd0-f3dccc884201 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.039010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baf39f-5483-46a9-9cd0-f3dccc884201 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.039727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baf39f-5483-46a9-9cd0-f3dccc884201 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.041106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 429/853] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:38.046413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b416-7c8c-406b-ac9d-cf5f597b34dd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.046869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b416-7c8c-406b-ac9d-cf5f597b34dd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.047152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c682b416-7c8c-406b-ac9d-cf5f597b34dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.065942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-c682b416-7c8c-406b-ac9d-cf5f597b34dd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:38.066862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 425/854] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:00:38.068001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c682b416-7c8c-406b-ac9d-cf5f597b34dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:38.072991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0f219-98d3-459d-8604-05bfb255eeb7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.148611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 430/855] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:38.149747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682b416-7c8c-406b-ac9d-cf5f597b34dd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.169619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0f219-98d3-459d-8604-05bfb255eeb7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.203990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cab260-52c2-45aa-8791-78144867f68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.204707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cab260-52c2-45aa-8791-78144867f68b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.204985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cab260-52c2-45aa-8791-78144867f68b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.205251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cab260-52c2-45aa-8791-78144867f68b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.215997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cab260-52c2-45aa-8791-78144867f68b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-b4cab260-52c2-45aa-8791-78144867f68b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:38.224041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:38.224956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:38.225843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 426/856] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:38.230822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cab260-52c2-45aa-8791-78144867f68b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.257441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c682b416-7c8c-406b-ac9d-cf5f597b34dd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.268455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:38.271040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:38.292894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.293687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.293967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.294239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.304084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:38.305522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:38.323330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:38.324263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:38.325172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:38.325172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:38.325172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:38.325172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:38.325172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 431/857] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:38.325462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.342946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cab260-52c2-45aa-8791-78144867f68b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.379351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c35dae-3899-4e08-a2ba-a247f70a8ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.380023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c35dae-3899-4e08-a2ba-a247f70a8ff8 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.380298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c35dae-3899-4e08-a2ba-a247f70a8ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.380569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c35dae-3899-4e08-a2ba-a247f70a8ff8 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.381406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c35dae-3899-4e08-a2ba-a247f70a8ff8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:38.383022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c35dae-3899-4e08-a2ba-a247f70a8ff8 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:38.398533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 427/858] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:00:38 2022] POST /identity/v3/services => generated 251 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:38.402178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c35dae-3899-4e08-a2ba-a247f70a8ff8 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.431946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66f914c-31e6-402f-8bd3-8e1ddf567b6a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.478870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.479954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.480980 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.481411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.482655 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:38.483221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:38.485981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:38.497753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 432/859] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:38.503369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.515620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:38.523569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c35dae-3899-4e08-a2ba-a247f70a8ff8 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.540483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:38.577694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.579290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.580096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.580521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.581413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:38.581862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:38.584059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:38.600086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 428/860] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:38.605417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.641263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c7804b-937f-450e-830d-df66352409d6 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.678199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.679467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.679942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.680426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.690997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:38.692976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:38.711145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:38.712085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:38.712987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:38.712987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:38.712987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:38.712987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:38.715211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 433/861] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:38.720667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.738056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539830b3-e0b7-478b-a990-e6cbf70e65c4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.777357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.778174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.778455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.778729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.786071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:38.787562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:38.802251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:38.803179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:38.804173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:38.804173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:38.804173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:38.804173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:38.804173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:38.804173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:38.804173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:38.804173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:38.807367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 429/862] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:38.809541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.821786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:38.829492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:38.858275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045bf43a-fb89-46d2-ab6e-d527f12bfe6f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.901725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.902647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.903076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.904058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.905088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:38.905688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:38.907728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:38.920354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f881e-2f7d-432b-923b-e39c51e2e4f5 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:38.931330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 434/863] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:38.933339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:38.968435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:38.970688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:38.971193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:38.971721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:38.972739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:38.973256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:38.974925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:38.991993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 430/864] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:38.998593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:39.044942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085c1d23-1ac1-453a-a923-863db88b99b5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:39.081166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:39.081988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:39.082929 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:39.083206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:39.083506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:39.086006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:39.093554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:39.095944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:39.107723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47177c62-b0f8-4824-b8c5-8c43a1fcec9f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:39.111844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:39.112784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:39.113770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:39.117695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 435/865] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:39.119143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:39.148771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:39.149821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:39.150305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:39.150777 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:39.151887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=569782752acb4085ad742330af999bc2, user.id=0c8203a8a7464c3bac93e5d42808c00f) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:39.152469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:39.154186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:39.167580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 431/866] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:38 2022] GET /identity/v3/role_assignments?user.id=0c8203a8a7464c3bac93e5d42808c00f&scope.project.id=569782752acb4085ad742330af999bc2&role.id=6dca675ade814980a252f800fb4fa3ad => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:39.229598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a284080-64b7-4101-8ba3-3e162d2ae5f1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:39.268456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe85b408-2cf9-47fa-8650-3ee690cb1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:39.269161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85b408-2cf9-47fa-8650-3ee690cb1e97 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:39.269440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85b408-2cf9-47fa-8650-3ee690cb1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:39.269707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe85b408-2cf9-47fa-8650-3ee690cb1e97 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:39.270600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85b408-2cf9-47fa-8650-3ee690cb1e97 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:39.270908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85b408-2cf9-47fa-8650-3ee690cb1e97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:39.272508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe85b408-2cf9-47fa-8650-3ee690cb1e97 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:39.299520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 436/867] 10.222.0.69 () {56 vars in 1149 bytes} [Fri May 20 09:00:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:39.301009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:39.415978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346abb4f-aa9a-4576-86e1-9f37e00773c9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:39.450947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40d34c-a7cd-488e-919d-f796c45c8733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:39.452014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40d34c-a7cd-488e-919d-f796c45c8733 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:39.452489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40d34c-a7cd-488e-919d-f796c45c8733 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:39.452952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c40d34c-a7cd-488e-919d-f796c45c8733 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:39.464724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea0f33-f0c7-40d1-9bd5-9b288b3331f6 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:39.465032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea0f33-f0c7-40d1-9bd5-9b288b3331f6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:39.465272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea0f33-f0c7-40d1-9bd5-9b288b3331f6 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:39.467018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:39.468519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 437/868] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:39.473577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8b6ad-d501-4d40-bbf2-008960e1ee9b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:39.474124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8b6ad-d501-4d40-bbf2-008960e1ee9b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:39.474407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c8b6ad-d501-4d40-bbf2-008960e1ee9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:39.492536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e0c8b6ad-d501-4d40-bbf2-008960e1ee9b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:39.493712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e0c8b6ad-d501-4d40-bbf2-008960e1ee9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:39.544382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40d34c-a7cd-488e-919d-f796c45c8733 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:39.545308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:39.548580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c40d34c-a7cd-488e-919d-f796c45c8733 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:39.572110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 438/869] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:39 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:39.595508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8b6ad-d501-4d40-bbf2-008960e1ee9b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:39.676395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 432/870] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:39 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 20 09:00:39.678378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40d34c-a7cd-488e-919d-f796c45c8733 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:39.702672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c8b6ad-d501-4d40-bbf2-008960e1ee9b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:39.732408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:39.742278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0be94b-93b6-449e-8065-e803e176f3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:39.742960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0be94b-93b6-449e-8065-e803e176f3dc None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:39.743240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0be94b-93b6-449e-8065-e803e176f3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:39.743536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0be94b-93b6-449e-8065-e803e176f3dc None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:39.744282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0be94b-93b6-449e-8065-e803e176f3dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:39.745664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0be94b-93b6-449e-8065-e803e176f3dc None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:39.768754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 439/871] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:00:39 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:00:39.807517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c40d34c-a7cd-488e-919d-f796c45c8733 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:39.859624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:39.860492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:39.860777 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:39.861051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:39.873598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:39.875143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:39.894987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:39.895971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:39.897339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:39.897339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:39.897339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:39.897339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:39.897339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 433/872] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:39.902449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0be94b-93b6-449e-8065-e803e176f3dc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:39.961072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:40.011763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0be94b-93b6-449e-8065-e803e176f3dc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:40.046790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:40.047482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:40.047816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:40.048091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:40.048871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:40.049178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:40.050555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:40.063619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 440/873] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:40.069237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:40.164644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:40.177994 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fdedd8-45f0-42fa-9995-44654daa878f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:40.214703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:40.215558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:40.215837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:40.216111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:40.223419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:40.224895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:40.238958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:40.239971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:40.241816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 434/874] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:40.246248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:40.347897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38481de-e063-45c3-93ef-efdbe84c2937 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:40.349384 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:40.384576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c98c942-0131-4f5c-8a90-67eadcc82bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:40.385262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98c942-0131-4f5c-8a90-67eadcc82bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:40.385542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98c942-0131-4f5c-8a90-67eadcc82bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:40.385815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c98c942-0131-4f5c-8a90-67eadcc82bc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:40.386652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98c942-0131-4f5c-8a90-67eadcc82bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:40.386963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98c942-0131-4f5c-8a90-67eadcc82bc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:40.388497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c98c942-0131-4f5c-8a90-67eadcc82bc6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:40.401783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 441/875] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:40.407675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:40.518373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef9ca8f-dd19-443c-805e-ebf9063971e3 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:40.556432 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4a5fff-4154-4851-9788-d1d131cf78c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:40.557095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a5fff-4154-4851-9788-d1d131cf78c5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:40.557378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a5fff-4154-4851-9788-d1d131cf78c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:40.557656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4a5fff-4154-4851-9788-d1d131cf78c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:40.558562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a5fff-4154-4851-9788-d1d131cf78c5 None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=6cd71c35c4224658aa2ab0b81f0f4b97) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:40.558879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a5fff-4154-4851-9788-d1d131cf78c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:40.560356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4a5fff-4154-4851-9788-d1d131cf78c5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:40.566785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 435/876] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:40 2022] GET /identity/v3/role_assignments?user.id=6cd71c35c4224658aa2ab0b81f0f4b97&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:40.687716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639bde4-5d13-4743-bbf9-24339203363f None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:40.688235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639bde4-5d13-4743-bbf9-24339203363f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:40.688235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1639bde4-5d13-4743-bbf9-24339203363f None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:40.689827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 442/877] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:40.697205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd782-6528-4d21-9672-8b80ebf6e379 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:40.697763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd782-6528-4d21-9672-8b80ebf6e379 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:40.697999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd782-6528-4d21-9672-8b80ebf6e379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:40.716779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-857cd782-6528-4d21-9672-8b80ebf6e379 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:40.717994 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-857cd782-6528-4d21-9672-8b80ebf6e379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:40.727212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:40.792145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 436/878] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:40 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:40.814132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639bde4-5d13-4743-bbf9-24339203363f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:40.923530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1639bde4-5d13-4743-bbf9-24339203363f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:40.958447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:40.959178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:40.959488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:40.959888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:40.960684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:40.962064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:40.974087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:40.975151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:40.976063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 443/879] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:40.983493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd782-6528-4d21-9672-8b80ebf6e379 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:41.091205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd782-6528-4d21-9672-8b80ebf6e379 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:41.126207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:41.126943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:41.127224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:41.127573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:41.128298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:41.128604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:41.130097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:41.135555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 437/880] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:41.141153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:41.171946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:41.248760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16939e01-ae24-4b66-a1c0-2c9d1b571477 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:41.282982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:41.283676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:41.283955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:41.284226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:41.284942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:41.285245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:41.286632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:41.292418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 444/881] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:41 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:41.300673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:41.311988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:41.407949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15eb9d-6839-4697-b9b6-d1dc9bbd42f9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:41.442144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c3441-f8f0-4576-bc75-db912b7e4446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:41.442810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c3441-f8f0-4576-bc75-db912b7e4446 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:41.443092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c3441-f8f0-4576-bc75-db912b7e4446 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:41.443407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c3441-f8f0-4576-bc75-db912b7e4446 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:41.444322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c3441-f8f0-4576-bc75-db912b7e4446 None admin] RBAC: Adding query filter params (interface=public, service_id=a05de27b850d4995bde368cc4465cd35, region_id=RegionOne) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:41.444635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c3441-f8f0-4576-bc75-db912b7e4446 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:41.446014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c3441-f8f0-4576-bc75-db912b7e4446 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:41.453996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 438/882] 10.222.0.69 () {56 vars in 1251 bytes} [Fri May 20 09:00:41 2022] GET /identity/v3/endpoints?service_id=a05de27b850d4995bde368cc4465cd35&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:41.460561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:41.494230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d7cc0-4c05-4fbc-a2a6-93b0fbc20aad None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:41.494537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d7cc0-4c05-4fbc-a2a6-93b0fbc20aad None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:41.494800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d7cc0-4c05-4fbc-a2a6-93b0fbc20aad None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:41.496394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 439/883] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:41.501697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdf742-5e7f-4851-9598-e8037603bb91 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:41.501926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdf742-5e7f-4851-9598-e8037603bb91 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:41.502199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bdf742-5e7f-4851-9598-e8037603bb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:41.520852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a5bdf742-5e7f-4851-9598-e8037603bb91 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:41.522437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a5bdf742-5e7f-4851-9598-e8037603bb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:41.571638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:41.582301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9070b7-8aff-4d65-8ef5-d315fc477663 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:41.593962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:41.607525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 440/884] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:41.620693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248bab9-e796-4d5e-8c32-3201e366a5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:41.621617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248bab9-e796-4d5e-8c32-3201e366a5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:41.622052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdf742-5e7f-4851-9598-e8037603bb91 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:41.622293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248bab9-e796-4d5e-8c32-3201e366a5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:41.622885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248bab9-e796-4d5e-8c32-3201e366a5c0 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:41.623778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248bab9-e796-4d5e-8c32-3201e366a5c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:41.625652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248bab9-e796-4d5e-8c32-3201e366a5c0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:41.632552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 445/885] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:00:41 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:00:41.731936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bdf742-5e7f-4851-9598-e8037603bb91 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:41.774337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:41.774337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:41.774337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:41.774337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:41.774793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:41.774793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:41.785516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:41.786559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:41.787430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 441/886] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:41.793648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248bab9-e796-4d5e-8c32-3201e366a5c0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:41.905636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248bab9-e796-4d5e-8c32-3201e366a5c0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:41.944095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d262c0-0c4a-49df-8f16-35e2583c23c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:41.944785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d262c0-0c4a-49df-8f16-35e2583c23c6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:41.945111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d262c0-0c4a-49df-8f16-35e2583c23c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:41.945394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d262c0-0c4a-49df-8f16-35e2583c23c6 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:41.946137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d262c0-0c4a-49df-8f16-35e2583c23c6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:41.946451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d262c0-0c4a-49df-8f16-35e2583c23c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:41.947923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d262c0-0c4a-49df-8f16-35e2583c23c6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:41.953838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 446/887] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:41.959992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:42.027051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:42.066139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc2a7b-6bb3-41dc-b23f-f888d49a0144 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:42.100798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ad4b6-7623-4f52-bbad-349af831be34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:42.101516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad4b6-7623-4f52-bbad-349af831be34 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:42.101804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad4b6-7623-4f52-bbad-349af831be34 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:42.102079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ad4b6-7623-4f52-bbad-349af831be34 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:42.102796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ad4b6-7623-4f52-bbad-349af831be34 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:42.103105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ad4b6-7623-4f52-bbad-349af831be34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:42.104524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8ad4b6-7623-4f52-bbad-349af831be34 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:42.109541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 442/888] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:41 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:42.124677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56b9fa-0e7f-4bdc-8125-1577b9362cf2 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:42.125037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56b9fa-0e7f-4bdc-8125-1577b9362cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:42.125230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b56b9fa-0e7f-4bdc-8125-1577b9362cf2 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:42.128353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 447/889] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:42.133214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f34fbc-9fce-4392-b57e-c652255798d8 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:42.133517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f34fbc-9fce-4392-b57e-c652255798d8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:42.133770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f34fbc-9fce-4392-b57e-c652255798d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:42.151579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e8f34fbc-9fce-4392-b57e-c652255798d8 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:42.152789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e8f34fbc-9fce-4392-b57e-c652255798d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:42.231538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 443/890] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:42 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:42.253905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56b9fa-0e7f-4bdc-8125-1577b9362cf2 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:42.274910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:42.368797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b56b9fa-0e7f-4bdc-8125-1577b9362cf2 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:42.403247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:42.404086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:42.404361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:42.404628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:42.405286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:42.406712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:42.419923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:42.420915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:42.421754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 448/891] 10.222.0.69 () {56 vars in 1132 bytes} [Fri May 20 09:00:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:42.427362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f34fbc-9fce-4392-b57e-c652255798d8 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:42.492137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:42.535111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f34fbc-9fce-4392-b57e-c652255798d8 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:42.570204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:42.570885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:42.571168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:42.571479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:42.572205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:42.572517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:42.574042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:42.579436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 444/892] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:42.584939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:42.677397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:42.697323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e995c3-2e66-4532-9195-fcc0f5e1f778 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:42.730480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:42.731188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:42.731497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:42.731819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:42.732528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:42.732831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:42.734205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:42.743361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 449/893] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:42 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:42.745305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:42.853374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2959486-c33f-4728-ad62-19f7af917097 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:42.887108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:42.901679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ef3b4-3f40-41fe-a0fb-9d112f0d07d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:42.902356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef3b4-3f40-41fe-a0fb-9d112f0d07d4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:42.902631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef3b4-3f40-41fe-a0fb-9d112f0d07d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:42.902903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ef3b4-3f40-41fe-a0fb-9d112f0d07d4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:42.903745 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ef3b4-3f40-41fe-a0fb-9d112f0d07d4 None admin] RBAC: Adding query filter params (interface=public, service_id=061cdd5b0558403a95353974555ea3b5, region_id=RegionOne) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:42.904053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ef3b4-3f40-41fe-a0fb-9d112f0d07d4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:42.905464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000ef3b4-3f40-41fe-a0fb-9d112f0d07d4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:42.917472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 445/894] 10.222.0.69 () {56 vars in 1251 bytes} [Fri May 20 09:00:42 2022] GET /identity/v3/endpoints?service_id=061cdd5b0558403a95353974555ea3b5&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:42.918835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:43.034768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ef2df-b651-47af-9a16-d0205ca071ba None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:43.065200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569640-5b62-40c5-9b53-a739611c7796 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:43.065484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569640-5b62-40c5-9b53-a739611c7796 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:43.065762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4569640-5b62-40c5-9b53-a739611c7796 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:43.067401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 446/895] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:43.069973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa7c3a-7ebd-4b42-a3de-2663568257ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:43.070838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa7c3a-7ebd-4b42-a3de-2663568257ed None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:43.071314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa7c3a-7ebd-4b42-a3de-2663568257ed None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:43.071760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fa7c3a-7ebd-4b42-a3de-2663568257ed None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:43.072216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766594c0-94fe-4b45-bb95-3dc4a7bd5f2a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:43.072496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766594c0-94fe-4b45-bb95-3dc4a7bd5f2a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:43.072768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766594c0-94fe-4b45-bb95-3dc4a7bd5f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:43.073205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa7c3a-7ebd-4b42-a3de-2663568257ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:43.074753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fa7c3a-7ebd-4b42-a3de-2663568257ed None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:43.081064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 450/896] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:00:42 2022] GET /identity/v3/services => generated 1268 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:00:43.089609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-766594c0-94fe-4b45-bb95-3dc4a7bd5f2a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:43.090835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-766594c0-94fe-4b45-bb95-3dc4a7bd5f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:43.169981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 447/897] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:43.192872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa7c3a-7ebd-4b42-a3de-2663568257ed None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:43.206447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:43.302656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fa7c3a-7ebd-4b42-a3de-2663568257ed None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:43.340007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0a027-f758-4bfc-8ad4-c3c277c33993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:43.341074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0a027-f758-4bfc-8ad4-c3c277c33993 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:43.341523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0a027-f758-4bfc-8ad4-c3c277c33993 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:43.341951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0a027-f758-4bfc-8ad4-c3c277c33993 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:43.352643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e0a027-f758-4bfc-8ad4-c3c277c33993 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-44e0a027-f758-4bfc-8ad4-c3c277c33993 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:43.360961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:43.362118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:00:43.363193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:43.365649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 451/898] 10.222.0.69 () {56 vars in 1120 bytes} [Fri May 20 09:00:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:43.371237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766594c0-94fe-4b45-bb95-3dc4a7bd5f2a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:43.474139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766594c0-94fe-4b45-bb95-3dc4a7bd5f2a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:43.491767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:43.510071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:43.511163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:43.511480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:43.511758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:43.512529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:43.512840 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:43.514249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:43.523818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 448/899] 10.222.0.69 () {56 vars in 1115 bytes} [Fri May 20 09:00:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:43.528967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0a027-f758-4bfc-8ad4-c3c277c33993 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:43.605196 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:43.637148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0a027-f758-4bfc-8ad4-c3c277c33993 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:43.674035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:43.674823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:43.675098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:43.675395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:43.686409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:43.687945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:43.705098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:43.707177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:43.709283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:43.709283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:43.709283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:43.709283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:43.709283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 452/900] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:43.712280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:43.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7865ad05-7c6a-4e78-b3d8-459b1d606c8a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:43.862505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa6997b-17fc-40bd-965e-ce89feb13236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:43.863521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6997b-17fc-40bd-965e-ce89feb13236 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:43.863973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6997b-17fc-40bd-965e-ce89feb13236 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:43.864416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa6997b-17fc-40bd-965e-ce89feb13236 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:43.865331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6997b-17fc-40bd-965e-ce89feb13236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:43.865844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6997b-17fc-40bd-965e-ce89feb13236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:43.867422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa6997b-17fc-40bd-965e-ce89feb13236 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:43.880428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 449/901] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:43.886242 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:43.989490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa2f5d-2098-40d7-b7f5-3029e6f4c44c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:44.012802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec099-4415-4b9c-a1fd-664bdc28def7 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.013100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec099-4415-4b9c-a1fd-664bdc28def7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.013505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eec099-4415-4b9c-a1fd-664bdc28def7 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.014184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:44.015420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 450/902] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:44.020602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df998e-823d-460c-884b-d6a865afdad6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.020964 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df998e-823d-460c-884b-d6a865afdad6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.021320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84df998e-823d-460c-884b-d6a865afdad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.029399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:44.030468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.030928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.031420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.038830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-84df998e-823d-460c-884b-d6a865afdad6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:44.040476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-84df998e-823d-460c-884b-d6a865afdad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:44.041128 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:44.042992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:44.062523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:44.063528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:44.064475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:44.065026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 453/903] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:00:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:44.072806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:44.079272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:44.123576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 451/904] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:44.143139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df998e-823d-460c-884b-d6a865afdad6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:44.195494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ab136-75d7-4792-a1b5-252cf7bdd730 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:44.231449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:44.232123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.232404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.232680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.233519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:44.233826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:44.235431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:44.259778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 454/905] 10.222.0.69 () {56 vars in 1149 bytes} [Fri May 20 09:00:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:44.261402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:44.272029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84df998e-823d-460c-884b-d6a865afdad6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:44.311564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:44.313287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:44.314342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.314837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.315323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.316199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:44.317798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:44.330292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:44.331330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 20 09:00:44.332176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:44.335044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 452/906] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:44.341493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:44.379196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:44.387183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bd2cd5-5e95-4f0b-a111-15dd6e48d34f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:44.430517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b89cb30-37c7-4f88-80f5-87e2e118a708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:44.431579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89cb30-37c7-4f88-80f5-87e2e118a708 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.432068 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89cb30-37c7-4f88-80f5-87e2e118a708 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.432541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b89cb30-37c7-4f88-80f5-87e2e118a708 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.463874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66014b1a-e323-4055-a15c-c931d6cf5bf3 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:44.500410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8b79c-c801-474c-b3a6-a1ced311e81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:44.501076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8b79c-c801-474c-b3a6-a1ced311e81f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.501357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8b79c-c801-474c-b3a6-a1ced311e81f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.501741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8b79c-c801-474c-b3a6-a1ced311e81f None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.502460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8b79c-c801-474c-b3a6-a1ced311e81f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:44.502764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8b79c-c801-474c-b3a6-a1ced311e81f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:44.504183 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8b79c-c801-474c-b3a6-a1ced311e81f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:44.518437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 453/907] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:44.518622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89cb30-37c7-4f88-80f5-87e2e118a708 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:44.518722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de834-9366-4a66-8460-d8f846085b61 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.518722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de834-9366-4a66-8460-d8f846085b61 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.518722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2de834-9366-4a66-8460-d8f846085b61 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.518913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b89cb30-37c7-4f88-80f5-87e2e118a708 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:44.518985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 454/908] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:44.518985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de834-9366-4a66-8460-d8f846085b61 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:44.556577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:44.615717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:44.618448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 455/909] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:44 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:00:44.619951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b86354-7586-44c3-9909-2c528e8180de None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.620432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b86354-7586-44c3-9909-2c528e8180de None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.620910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b86354-7586-44c3-9909-2c528e8180de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.627370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2de834-9366-4a66-8460-d8f846085b61 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:44.640190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-76b86354-7586-44c3-9909-2c528e8180de None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:44.641322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-76b86354-7586-44c3-9909-2c528e8180de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:44.662251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:44.662925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.663203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.663506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.664575 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:44.664911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:44.666494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:44.672625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 455/910] 10.222.0.69 () {56 vars in 1114 bytes} [Fri May 20 09:00:44 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:44.673649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:44.716877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 456/911] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:44.718035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b86354-7586-44c3-9909-2c528e8180de None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:44.792423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:44.804836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd17161-f52c-4a76-bce2-93e60c9dfe2e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:44.826648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b86354-7586-44c3-9909-2c528e8180de None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:44.837409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:44.838226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.838503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.838776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.849152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:44.852499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:44.863628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849a422-9246-4eb9-849c-e0e2abeb7d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:44.864570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a422-9246-4eb9-849c-e0e2abeb7d56 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:44.865043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a422-9246-4eb9-849c-e0e2abeb7d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:44.865491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a422-9246-4eb9-849c-e0e2abeb7d56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:44.866478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849a422-9246-4eb9-849c-e0e2abeb7d56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:44.868102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849a422-9246-4eb9-849c-e0e2abeb7d56 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:44.870149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:44.871034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:44.872030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:44.872030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:44.872030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:44.872030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:44.876028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 456/912] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:44.876626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:44.891967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:44.895612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 457/913] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:00:44 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:44.896454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849a422-9246-4eb9-849c-e0e2abeb7d56 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:45.007826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894b7dd6-2dfb-4f55-a440-89d171b14327 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:45.025050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849a422-9246-4eb9-849c-e0e2abeb7d56 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:45.054491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:45.057930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f762f0a-2912-467e-b066-4c05e587c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:45.058621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f762f0a-2912-467e-b066-4c05e587c26d None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.058897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f762f0a-2912-467e-b066-4c05e587c26d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.059166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f762f0a-2912-467e-b066-4c05e587c26d None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.059975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f762f0a-2912-467e-b066-4c05e587c26d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:45.061382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f762f0a-2912-467e-b066-4c05e587c26d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:45.076522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:45.077763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.078287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.078846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.079440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 457/914] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:00:44 2022] POST /identity/v3/services => generated 241 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:45.080120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:45.080616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:45.082101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:45.101645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 458/915] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:45.107501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f762f0a-2912-467e-b066-4c05e587c26d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:45.200894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:45.245223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f762f0a-2912-467e-b066-4c05e587c26d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:45.282317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12742a82-8ae9-42f4-8ed7-e03484fe4907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:45.283126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12742a82-8ae9-42f4-8ed7-e03484fe4907 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.283432 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12742a82-8ae9-42f4-8ed7-e03484fe4907 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.283706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12742a82-8ae9-42f4-8ed7-e03484fe4907 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.291366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12742a82-8ae9-42f4-8ed7-e03484fe4907 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:45.292829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12742a82-8ae9-42f4-8ed7-e03484fe4907 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-12742a82-8ae9-42f4-8ed7-e03484fe4907 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:45.307348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:45.308325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:45.309224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 458/916] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:45.314615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:45.378408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785d8f1-05b1-4079-a731-4c30e27e5c91 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.378693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785d8f1-05b1-4079-a731-4c30e27e5c91 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.378974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785d8f1-05b1-4079-a731-4c30e27e5c91 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.381851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 459/917] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:45.385991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f613f3-0a73-42e5-9882-2f4ae507cf39 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.386274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f613f3-0a73-42e5-9882-2f4ae507cf39 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.386547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f613f3-0a73-42e5-9882-2f4ae507cf39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.411775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-69f613f3-0a73-42e5-9882-2f4ae507cf39 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:45.413265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-69f613f3-0a73-42e5-9882-2f4ae507cf39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:45.430064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e93a4-56ae-4d54-9b67-d9e475d230e0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:45.472895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:45.473795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.474223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.474597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.475805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:45.476682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:45.478531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:45.497072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 459/918] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:45.506538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:45.506741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 460/919] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:45.526831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f613f3-0a73-42e5-9882-2f4ae507cf39 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:45.554058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:45.594529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:45.614294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8536d5-b53e-4659-962f-7174a5be2ed0 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:45.641943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f613f3-0a73-42e5-9882-2f4ae507cf39 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:45.653843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:45.654990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.655618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.655956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.657129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] RBAC: Adding query filter params (role.id=8bdf6fdbab194849a9ac0170382b070b, scope.project.id=569782752acb4085ad742330af999bc2, user.id=6cd71c35c4224658aa2ab0b81f0f4b97) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:45.657977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:45.659869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:45.668349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 460/920] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:45 2022] GET /identity/v3/role_assignments?user.id=6cd71c35c4224658aa2ab0b81f0f4b97&scope.project.id=569782752acb4085ad742330af999bc2&role.id=8bdf6fdbab194849a9ac0170382b070b => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:45.684907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:45.685924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.686305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.686675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.687528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:45.689106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:45.702697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:45.703756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:45.705104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 461/921] 10.222.0.69 () {56 vars in 1132 bytes} [Fri May 20 09:00:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:45.710707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:45.827360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972da0b-1e56-4123-8e82-c30f6ae310e1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:45.835835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:45.865551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:45.866260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:45.866585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:45.866888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:45.867658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:45.867969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:45.869359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:45.874940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 461/922] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:45.880721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:45.985507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d948304f-857c-4447-b8a0-1633e1f4bd11 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:46.008884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:46.020949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e2e9a0-5dad-4f34-860f-3e6017681cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:46.022003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e9a0-5dad-4f34-860f-3e6017681cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:46.022724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e9a0-5dad-4f34-860f-3e6017681cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:46.023198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e2e9a0-5dad-4f34-860f-3e6017681cd4 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:46.024138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2e9a0-5dad-4f34-860f-3e6017681cd4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:46.024654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2e9a0-5dad-4f34-860f-3e6017681cd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:46.026240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e2e9a0-5dad-4f34-860f-3e6017681cd4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:46.032037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 462/923] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:46.039493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:46.147725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1756b59-76ac-49c4-9773-a1134c6f9188 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:46.182557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3061b0a9-b8e7-4ce8-8bcb-9a7a66d6f297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:46.183219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061b0a9-b8e7-4ce8-8bcb-9a7a66d6f297 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:46.183531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061b0a9-b8e7-4ce8-8bcb-9a7a66d6f297 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:46.183804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3061b0a9-b8e7-4ce8-8bcb-9a7a66d6f297 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:46.184564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061b0a9-b8e7-4ce8-8bcb-9a7a66d6f297 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:46.185949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3061b0a9-b8e7-4ce8-8bcb-9a7a66d6f297 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:46.209129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 462/924] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:00:46 2022] POST /identity/v3/endpoints => generated 347 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:47.331147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc5e16-e25c-4d30-946a-c3889e9f61dd None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.332365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc5e16-e25c-4d30-946a-c3889e9f61dd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.332878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dc5e16-e25c-4d30-946a-c3889e9f61dd None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.334815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 463/925] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:47.341041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df56a46-d872-44e9-9ecc-9850877956cc None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.342048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df56a46-d872-44e9-9ecc-9850877956cc None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.342534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df56a46-d872-44e9-9ecc-9850877956cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.360617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4df56a46-d872-44e9-9ecc-9850877956cc None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:47.362142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4df56a46-d872-44e9-9ecc-9850877956cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:47.383125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:47.447361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 463/926] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:47.463715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc5e16-e25c-4d30-946a-c3889e9f61dd None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:47.487069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e243e7-a7f5-4b8c-b13d-c2b68af44b24 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.487421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e243e7-a7f5-4b8c-b13d-c2b68af44b24 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.487651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e243e7-a7f5-4b8c-b13d-c2b68af44b24 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.489590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 464/927] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:47.495866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48abb9-e225-471c-96b7-459e5c2faf49 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.496151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48abb9-e225-471c-96b7-459e5c2faf49 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.496476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48abb9-e225-471c-96b7-459e5c2faf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.517476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-ce48abb9-e225-471c-96b7-459e5c2faf49 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:47.519239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ce48abb9-e225-471c-96b7-459e5c2faf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:47.578841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dc5e16-e25c-4d30-946a-c3889e9f61dd None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:47.607468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 465/928] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:47.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:47.619435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.619936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.620413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.621319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:47.623030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:47.624626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING keystone.server.flask.common [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 20 09:00:47.630030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48abb9-e225-471c-96b7-459e5c2faf49 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:47.646482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.sql.core [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 20 09:00:47.646482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 20 09:00:47.646482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [parameters: {'id': '2d1d4b3cc59f4a189a86fceb0cd327aa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 20 09:00:47.646482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80637) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2d1d4b3cc59f4a189a86fceb0cd327aa.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2d1d4b3cc59f4a189a86fceb0cd327aa. May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.dialect.do_execute( May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result = self._query(query) May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application conn.query(q) May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 20 09:00:47.655320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result.read() May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application packet.raise_for_error() May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 20 09:00:47.656148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return project_ref.to_dict() May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application next(self.gen) May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application yield resource May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application next(self.gen) May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.session.rollback() May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.force_reraise() May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise self.value May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application session.commit() May 20 09:00:47.657098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._prepare_impl() May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.session.flush() May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._flush(objects) May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application compat.raise_( May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application flush_context.execute() May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 20 09:00:47.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rec.execute(self) May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application _emit_insert_statements( May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application c = connection._execute_20( May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ret = self._execute_context( May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 20 09:00:47.658943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.dialect.do_execute( May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result = self._query(query) May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application conn.query(q) May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result.read() May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 20 09:00:47.659790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application packet.raise_for_error() May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [parameters: {'id': '2d1d4b3cc59f4a189a86fceb0cd327aa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:47.660669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:47.661455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.Conflict( May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2d1d4b3cc59f4a189a86fceb0cd327aa. May 20 09:00:47.662400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:47.672389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 464/929] 10.222.0.69 () {60 vars in 1140 bytes} [Fri May 20 09:00:47 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:47.673406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:47.719410 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:47.725980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:47.744748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48abb9-e225-471c-96b7-459e5c2faf49 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:47.780419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:47.781153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.781429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.781697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.782355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:47.783880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:47.796190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952e67e-cf9c-460c-ac5e-7887555aafbf None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:47.796852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:47.797851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:47.798692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:47.803123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 466/930] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:47.804532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:47.837345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:47.838535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.838963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.839502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.846967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:47.848992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:47.864143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:47.865235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:47.866256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:47.866256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:47.866256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:47.866256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:47.866256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:47.866256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:47.866256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:47.866256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:47.873196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 465/931] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:47.874231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:47.911888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743e465-c716-474a-bcb1-f5e0cf17db31 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:47.945688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba0632-ffec-4649-8c28-69f1d5fad8a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:47.946454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba0632-ffec-4649-8c28-69f1d5fad8a1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.946730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba0632-ffec-4649-8c28-69f1d5fad8a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.947000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba0632-ffec-4649-8c28-69f1d5fad8a1 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.947742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba0632-ffec-4649-8c28-69f1d5fad8a1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:47.948043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba0632-ffec-4649-8c28-69f1d5fad8a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:47.949411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba0632-ffec-4649-8c28-69f1d5fad8a1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:47.954290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 467/932] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:47.955598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07525fb8-0c3d-49c8-9433-6cf4520aebe6 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:47.955877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07525fb8-0c3d-49c8-9433-6cf4520aebe6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:47.956150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07525fb8-0c3d-49c8-9433-6cf4520aebe6 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:47.957511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 468/933] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:47.962082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07525fb8-0c3d-49c8-9433-6cf4520aebe6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:47.974557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:47.988902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e65f9-90b2-40e3-9bb8-9221ab8edb19 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:47.990820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:48.025110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49869da7-50cc-4563-bafa-87608da39c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.026844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49869da7-50cc-4563-bafa-87608da39c3f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.027508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49869da7-50cc-4563-bafa-87608da39c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.027949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49869da7-50cc-4563-bafa-87608da39c3f None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.028732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49869da7-50cc-4563-bafa-87608da39c3f None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:48.029049 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49869da7-50cc-4563-bafa-87608da39c3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:48.030495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49869da7-50cc-4563-bafa-87608da39c3f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:48.051634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 466/934] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:47 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:48.053101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d994b-5465-4a0a-9a83-eddb4cd378ea None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.053419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d994b-5465-4a0a-9a83-eddb4cd378ea None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.053704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51d994b-5465-4a0a-9a83-eddb4cd378ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.071968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e51d994b-5465-4a0a-9a83-eddb4cd378ea None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:48.073518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e51d994b-5465-4a0a-9a83-eddb4cd378ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:48.081872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07525fb8-0c3d-49c8-9433-6cf4520aebe6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:48.124102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.125414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.125414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.125630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.126226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:48.126580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:48.128386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:48.135076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 469/935] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:48.140402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.167545 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 467/936] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:48.188507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d994b-5465-4a0a-9a83-eddb4cd378ea None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.239831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:48.270983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c107479-3c95-4b64-b8cf-db9a7111bf9d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:48.294325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:48.311800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51d994b-5465-4a0a-9a83-eddb4cd378ea None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:48.322505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.323375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.323725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.324066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.324963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] RBAC: Adding query filter params (interface=public, service_id=7eaee7644d25417ab30bf5fe26d74915, region_id=RegionOne) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:48.325344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:48.326852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:48.333972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 470/937] 10.222.0.69 () {56 vars in 1251 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/endpoints?service_id=7eaee7644d25417ab30bf5fe26d74915&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:48.339580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.366288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d1ee-0e87-4e7d-8e7f-1c254b3581e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.366821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d1ee-0e87-4e7d-8e7f-1c254b3581e5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.367055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d1ee-0e87-4e7d-8e7f-1c254b3581e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.367357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f12d1ee-0e87-4e7d-8e7f-1c254b3581e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.379590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f12d1ee-0e87-4e7d-8e7f-1c254b3581e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-5f12d1ee-0e87-4e7d-8e7f-1c254b3581e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:48.387810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:48.388948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:00:48.389988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:48.392456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 468/938] 10.222.0.69 () {56 vars in 1120 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:48.399173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d1ee-0e87-4e7d-8e7f-1c254b3581e5 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.477953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fddaf-9376-41d6-a0fe-54c76c36922b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:48.507216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f12d1ee-0e87-4e7d-8e7f-1c254b3581e5 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:48.522538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff80d1-8b85-4ac7-981d-daeb0a9de641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.523493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff80d1-8b85-4ac7-981d-daeb0a9de641 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.523774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff80d1-8b85-4ac7-981d-daeb0a9de641 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.524046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff80d1-8b85-4ac7-981d-daeb0a9de641 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.524735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff80d1-8b85-4ac7-981d-daeb0a9de641 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:48.526097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff80d1-8b85-4ac7-981d-daeb0a9de641 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:48.536768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 471/939] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:00:48.536768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89992896-f0a2-4749-a9be-f444ac7a81e2 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.536768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89992896-f0a2-4749-a9be-f444ac7a81e2 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.536768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89992896-f0a2-4749-a9be-f444ac7a81e2 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.536768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 472/940] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:48 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:48.541567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb995c-bb7c-4d32-980a-2c436f76051b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.541860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb995c-bb7c-4d32-980a-2c436f76051b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.542124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb995c-bb7c-4d32-980a-2c436f76051b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.547646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:48.550342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.550916 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.551212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.555591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.558051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:48.558460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:48.560519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:48.563579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-90eb995c-bb7c-4d32-980a-2c436f76051b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:48.564444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-90eb995c-bb7c-4d32-980a-2c436f76051b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:48.575472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 469/941] 10.222.0.69 () {56 vars in 1115 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:48.578959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.607698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:48.649557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 473/942] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:48.671429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb995c-bb7c-4d32-980a-2c436f76051b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.709730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd587d1-d3f7-4b74-a75b-2c067b685ce4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:48.747187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.748060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.748344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.748616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.759205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:48.760728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:48.782482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:48.783466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:48.784361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:48.784361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:48.784361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:48.784361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:48.786427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 470/943] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:48.788634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb995c-bb7c-4d32-980a-2c436f76051b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:48.792743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.820752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:48.828393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.829781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.830452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.831200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.832122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:48.833790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:48.847574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:48.848623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:48.849462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:48.852611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 474/944] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:48.859625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.868679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:48.906601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4515a-2096-4c1b-8062-6bae69b4c249 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:48.945024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:48.946000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:48.946485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:48.946961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:48.947954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:48.948451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:48.950016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:48.962794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 471/945] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:48.969526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:48.992504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278969b4-fd90-4ff8-a83c-37c28044562b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:49.043899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:49.044566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:49.044843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:49.045113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:49.045848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:49.046131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:49.048149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:49.058017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 475/946] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:49.059031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:49.108908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bab815-c59d-43b0-8f05-2888ffc981a8 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:49.119516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:49.148884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:49.161357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:49.163136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:49.163651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:49.164646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:49.171192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727a4a95-7316-4385-87f8-a351b7df04a2 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:49.175383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:49.177016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:49.196558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:49.197698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:49.198867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:49.201939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 472/947] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:00:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:49.208575 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:49.214009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:49.214210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:49.214611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:49.214944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:49.216932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:49.217323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:49.218808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:49.231286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 476/948] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:49 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:49.330039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5843c27-e926-46b9-b7bd-106b8e622a6f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:49.371900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:49.373433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:49.373913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:49.374379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:49.375472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:49.375982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:49.377759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:49.399815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 473/949] 10.222.0.69 () {56 vars in 1149 bytes} [Fri May 20 09:00:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:49.405659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:49.514760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e51206-403a-4640-866d-d637c590f4f8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:49.539874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:49.551142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:49.551964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:49.552241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:49.552514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:49.564240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:49.565689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:49.582658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:49.583564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:49.584887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:49.584887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:49.584887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:49.584887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:49.584887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 477/950] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:49.589812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:49.636012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:49.691578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc27cf-d858-4195-bfc1-590f92960a1b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:49.728969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:49.729698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:49.729978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:49.730246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:49.731028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:49.731362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:49.732734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:49.744925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 474/951] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:49.750598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:49.861932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92842fec-74ba-449b-bbcd-ddd267ddadcc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:49.895865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:49.896938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:49.897414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:49.897826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:49.904897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:49.906519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:49.921124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:49.922415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:49.923614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:49.923614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:49.923614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:49.923614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:49.923614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:49.923614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:49.923614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:49.923614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:49.925781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 478/952] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:49.933120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:50.032877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53d821-3336-4f21-991d-c468e0e8d6b4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:50.035146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:50.067612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9872d8-ff1a-4747-9aea-4cbe199280bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:50.068282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9872d8-ff1a-4747-9aea-4cbe199280bd None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:50.068563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9872d8-ff1a-4747-9aea-4cbe199280bd None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:50.068874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9872d8-ff1a-4747-9aea-4cbe199280bd None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:50.069669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9872d8-ff1a-4747-9aea-4cbe199280bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:50.069977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9872d8-ff1a-4747-9aea-4cbe199280bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:50.071439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9872d8-ff1a-4747-9aea-4cbe199280bd None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:50.084047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 475/953] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:50.089892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:50.126352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:50.192894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad2216-332b-4fd2-bd84-8ddd575e51bc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:50.227671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a2343-adbd-4338-b104-1647e5bc0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:50.228988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2343-adbd-4338-b104-1647e5bc0b93 None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:50.229441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2343-adbd-4338-b104-1647e5bc0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:50.229894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a2343-adbd-4338-b104-1647e5bc0b93 None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/6cd71c35c4224658aa2ab0b81f0f4b97/roles/8bdf6fdbab194849a9ac0170382b070b` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:50.232510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a2343-adbd-4338-b104-1647e5bc0b93 None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=6cd71c35c4224658aa2ab0b81f0f4b97, role_id=8bdf6fdbab194849a9ac0170382b070b)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:50.234220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a2343-adbd-4338-b104-1647e5bc0b93 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:50.260272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 479/954] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:00:50 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/6cd71c35c4224658aa2ab0b81f0f4b97/roles/8bdf6fdbab194849a9ac0170382b070b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:00:50.426004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548915ef-2183-44b5-9130-ed73639a830b None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:50.426534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548915ef-2183-44b5-9130-ed73639a830b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:50.426534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548915ef-2183-44b5-9130-ed73639a830b None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:50.428256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 476/955] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:50.434115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dee538-a62f-4a87-aebd-6f7a4f884f5b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:50.434729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dee538-a62f-4a87-aebd-6f7a4f884f5b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:50.435302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dee538-a62f-4a87-aebd-6f7a4f884f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:50.454444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-98dee538-a62f-4a87-aebd-6f7a4f884f5b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:50.455685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-98dee538-a62f-4a87-aebd-6f7a4f884f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:50.533653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 480/956] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:50.557961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548915ef-2183-44b5-9130-ed73639a830b None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:50.659496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548915ef-2183-44b5-9130-ed73639a830b None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:50.699462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8b0b5-b031-4e69-9506-982fcb8aba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:50.700558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8b0b5-b031-4e69-9506-982fcb8aba28 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:50.701094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8b0b5-b031-4e69-9506-982fcb8aba28 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:50.701776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8b0b5-b031-4e69-9506-982fcb8aba28 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:50.702723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8b0b5-b031-4e69-9506-982fcb8aba28 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:50.704498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b8b0b5-b031-4e69-9506-982fcb8aba28 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:50.720752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.sql.core [None req-97b8b0b5-b031-4e69-9506-982fcb8aba28 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 20 09:00:50.720752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 20 09:00:50.720752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [parameters: {'user_id': '7b02790aaca6413b872afb794efff803', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 20 09:00:50.720752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80637) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-97b8b0b5-b031-4e69-9506-982fcb8aba28 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.dialect.do_execute( May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result = self._query(query) May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application conn.query(q) May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result.read() May 20 09:00:50.729509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application packet.raise_for_error() May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 20 09:00:50.730404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application next(self.gen) May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application yield resource May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application next(self.gen) May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.session.rollback() May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.force_reraise() May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise self.value May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application session.commit() May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 20 09:00:50.731410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._prepare_impl() May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.session.flush() May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._flush(objects) May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application compat.raise_( May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application flush_context.execute() May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rec.execute(self) May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 20 09:00:50.732436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application _emit_insert_statements( May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result = connection._execute_20( May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ret = self._execute_context( May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 20 09:00:50.733324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self.dialect.do_execute( May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result = self._query(query) May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application conn.query(q) May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application result.read() May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 20 09:00:50.734195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application packet.raise_for_error() May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [parameters: {'user_id': '7b02790aaca6413b872afb794efff803', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:50.736374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 20 09:00:50.738223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 20 09:00:50.740380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 20 09:00:50.740380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 20 09:00:50.740380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 20 09:00:50.740380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 20 09:00:50.740380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 20 09:00:50.740380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:50.742954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 477/957] 10.222.0.69 () {60 vars in 1131 bytes} [Fri May 20 09:00:50 2022] POST /identity/v3/users => generated 160 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:50.748566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dee538-a62f-4a87-aebd-6f7a4f884f5b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:50.777502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:50.865229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dee538-a62f-4a87-aebd-6f7a4f884f5b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:50.889053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3607f0-3a49-4d56-b1db-6b2a0ad84283 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:50.889365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3607f0-3a49-4d56-b1db-6b2a0ad84283 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:50.889611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3607f0-3a49-4d56-b1db-6b2a0ad84283 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:50.894776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 478/958] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:50.901005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135418-9cad-4cb3-9f46-4cb5edb57382 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:50.901237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135418-9cad-4cb3-9f46-4cb5edb57382 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:50.901482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71135418-9cad-4cb3-9f46-4cb5edb57382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:50.904306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:50.905103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:50.905389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:50.905665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:50.913392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:50.915170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:50.920403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-71135418-9cad-4cb3-9f46-4cb5edb57382 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:50.922182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-71135418-9cad-4cb3-9f46-4cb5edb57382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:50.930212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:50.931156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:50.932218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 481/959] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:00:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:50.937274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:50.947601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:51.011347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 479/960] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:51.036228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71135418-9cad-4cb3-9f46-4cb5edb57382 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:51.043382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:51.062520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8823d741-3c94-4ad0-acf6-10e92d5bf54b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:51.112199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:51.113440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:51.113724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:51.113997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:51.114815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:51.115235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:51.116867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:51.150929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71135418-9cad-4cb3-9f46-4cb5edb57382 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:51.171608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 482/961] 10.222.0.69 () {56 vars in 1101 bytes} [Fri May 20 09:00:50 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:51.193777 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:51.195035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:51.195557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:51.196057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:51.196958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:51.198767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:51.212595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:51.213751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:51.214612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:51.222986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 480/962] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:51.224843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:51.336143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dd628-4b46-40c0-8cf6-f8bb1ca6f7b9 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:51.380710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:51.381536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:51.381819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:51.382092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:51.382834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:51.383140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:51.389364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:51.390975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 483/963] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:51.397399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:51.425033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:51.511912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1afac31-1897-4c98-8aa4-b599706b3379 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:51.547142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a58f205-0cc3-4a53-b92b-f75dbf40e9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:51.547833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f205-0cc3-4a53-b92b-f75dbf40e9fa None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:51.548117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f205-0cc3-4a53-b92b-f75dbf40e9fa None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:51.548386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58f205-0cc3-4a53-b92b-f75dbf40e9fa None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:51.549098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58f205-0cc3-4a53-b92b-f75dbf40e9fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:51.549402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58f205-0cc3-4a53-b92b-f75dbf40e9fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:51.550765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a58f205-0cc3-4a53-b92b-f75dbf40e9fa None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:51.556432 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 481/964] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:51.562136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:51.573504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:51.581312 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe536f-6568-4d8e-b338-d7cfdc251dec None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:51.581649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe536f-6568-4d8e-b338-d7cfdc251dec None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:51.581875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbe536f-6568-4d8e-b338-d7cfdc251dec None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:51.583530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 482/965] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:00:51.588602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da7b54-cb99-447f-9e80-2e7bcb4da68a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:51.588827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da7b54-cb99-447f-9e80-2e7bcb4da68a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:51.589097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da7b54-cb99-447f-9e80-2e7bcb4da68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:51.606611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f8da7b54-cb99-447f-9e80-2e7bcb4da68a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:51.607817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f8da7b54-cb99-447f-9e80-2e7bcb4da68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:51.666099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90724b-118d-4190-82ef-a92ef0f3f258 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:51.681201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 483/966] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:51.700414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fd7fbc-8aff-4a60-b08f-22ee3cc69db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:51.701070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7fbc-8aff-4a60-b08f-22ee3cc69db6 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:51.701354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7fbc-8aff-4a60-b08f-22ee3cc69db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:51.701629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fd7fbc-8aff-4a60-b08f-22ee3cc69db6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:51.702396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd7fbc-8aff-4a60-b08f-22ee3cc69db6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:51.703328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da7b54-cb99-447f-9e80-2e7bcb4da68a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:51.703779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fd7fbc-8aff-4a60-b08f-22ee3cc69db6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:51.719854 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:51.739326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 484/967] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:00:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:00:51.834899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da7b54-cb99-447f-9e80-2e7bcb4da68a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:51.880688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d59947-c7fc-4390-a64a-2dd0a28037fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:51.881740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d59947-c7fc-4390-a64a-2dd0a28037fe None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:51.882168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d59947-c7fc-4390-a64a-2dd0a28037fe None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:51.882587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d59947-c7fc-4390-a64a-2dd0a28037fe None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:51.883518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d59947-c7fc-4390-a64a-2dd0a28037fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:51.885066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d59947-c7fc-4390-a64a-2dd0a28037fe None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:51.903528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 484/968] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:00:51 2022] POST /identity/v3/services => generated 242 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:00:52.563933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeb758-7ca4-49e7-a3c5-2adab7d3a0e7 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:52.564476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeb758-7ca4-49e7-a3c5-2adab7d3a0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:52.564476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbeb758-7ca4-49e7-a3c5-2adab7d3a0e7 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:52.565993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 485/969] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:52.571152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce1872-2180-4b3b-852c-99098dbb7291 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:52.571606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce1872-2180-4b3b-852c-99098dbb7291 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:52.571779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ce1872-2180-4b3b-852c-99098dbb7291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:52.589398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a0ce1872-2180-4b3b-852c-99098dbb7291 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:52.590574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a0ce1872-2180-4b3b-852c-99098dbb7291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:52.656036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:52.669653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 485/970] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:52 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:52.692407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbeb758-7ca4-49e7-a3c5-2adab7d3a0e7 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:52.755501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:52.803714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbeb758-7ca4-49e7-a3c5-2adab7d3a0e7 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:52.836552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad3820-82fa-44bf-b1ac-746262fc157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:52.837278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad3820-82fa-44bf-b1ac-746262fc157a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:52.837562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad3820-82fa-44bf-b1ac-746262fc157a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:52.837824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ad3820-82fa-44bf-b1ac-746262fc157a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:52.847804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ad3820-82fa-44bf-b1ac-746262fc157a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-39ad3820-82fa-44bf-b1ac-746262fc157a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:00:52.855923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:52.856835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:00:52.857680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:52.859587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 486/971] 10.222.0.69 () {56 vars in 1120 bytes} [Fri May 20 09:00:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:52.863157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce1872-2180-4b3b-852c-99098dbb7291 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:52.969722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ce1872-2180-4b3b-852c-99098dbb7291 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:53.005987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:53.007027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:53.007523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:53.007976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:53.008936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:53.009424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:53.010988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:53.023337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 486/972] 10.222.0.69 () {56 vars in 1115 bytes} [Fri May 20 09:00:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:53.026083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad3820-82fa-44bf-b1ac-746262fc157a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:53.131594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ad3820-82fa-44bf-b1ac-746262fc157a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:53.156775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:53.168782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:53.169582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:53.169864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:53.170140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:53.180751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:53.182210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:53.199455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:53.200347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:53.206002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:53.206002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:53.206002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:53.206002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:53.206002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 487/973] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:53.206477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:53.267517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:53.311210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d38c6-2858-456e-aa37-2b0ed08aa666 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:53.345513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:53.346172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:53.346449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:53.346726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:53.347477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:53.347783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:53.349162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:53.365911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 487/974] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:53.367132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:53.480463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a05e72-5cf7-4464-9f28-cd276de6ab6c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:53.514968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:53.515807 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:53.516077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:53.516359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:53.523717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:53.525155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:00:53.540214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:53.541185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:53.546837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 488/975] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:00:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:53.547984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:53.649412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8b2c0b-0b8e-421f-b904-c083fde72214 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:53.668310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:53.686625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346d4a3f-6546-4f09-996b-1509089a9b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:53.687501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d4a3f-6546-4f09-996b-1509089a9b6e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:53.687690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d4a3f-6546-4f09-996b-1509089a9b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:53.687943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346d4a3f-6546-4f09-996b-1509089a9b6e None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:53.688784 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d4a3f-6546-4f09-996b-1509089a9b6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:53.689089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d4a3f-6546-4f09-996b-1509089a9b6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:53.690559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346d4a3f-6546-4f09-996b-1509089a9b6e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:53.718041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 488/976] 10.222.0.69 () {56 vars in 1149 bytes} [Fri May 20 09:00:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:53.719323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:53.755274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:53.835497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60862132-d148-4fe1-b99a-37135c7a92d6 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:53.871641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722c5a3-b7f2-4a91-a6cd-573c4467c683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:53.872590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722c5a3-b7f2-4a91-a6cd-573c4467c683 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:53.873040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722c5a3-b7f2-4a91-a6cd-573c4467c683 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:53.873512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3722c5a3-b7f2-4a91-a6cd-573c4467c683 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:53.942012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722c5a3-b7f2-4a91-a6cd-573c4467c683 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:53.943893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3722c5a3-b7f2-4a91-a6cd-573c4467c683 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:53.983447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:53.986477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c291c-f2d5-4d69-bba4-5db2f68b37f9 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:53.986725 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c291c-f2d5-4d69-bba4-5db2f68b37f9 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:53.987029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7c291c-f2d5-4d69-bba4-5db2f68b37f9 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:53.988595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 489/977] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:53.993911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c3df-4c11-4fc6-8150-a81d01b6dff7 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:53.994224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c3df-4c11-4fc6-8150-a81d01b6dff7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:53.994438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b644c3df-4c11-4fc6-8150-a81d01b6dff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.011310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b644c3df-4c11-4fc6-8150-a81d01b6dff7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:54.012445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b644c3df-4c11-4fc6-8150-a81d01b6dff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:54.054473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 489/978] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:00:53 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 20 09:00:54.060208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722c5a3-b7f2-4a91-a6cd-573c4467c683 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.094985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 490/979] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:53 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:54.116932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c3df-4c11-4fc6-8150-a81d01b6dff7 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.171667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3722c5a3-b7f2-4a91-a6cd-573c4467c683 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.204652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302bfb3-194c-4267-b3bb-72265c4a9684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.205433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302bfb3-194c-4267-b3bb-72265c4a9684 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.205708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302bfb3-194c-4267-b3bb-72265c4a9684 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.205978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302bfb3-194c-4267-b3bb-72265c4a9684 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.208578 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:54.213775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b644c3df-4c11-4fc6-8150-a81d01b6dff7 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.217755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302bfb3-194c-4267-b3bb-72265c4a9684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.219243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302bfb3-194c-4267-b3bb-72265c4a9684 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.221556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-c302bfb3-194c-4267-b3bb-72265c4a9684 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:00:54.237993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:54.238902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:54.239959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:00:54.239959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:00:54.239959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:00:54.239959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:54.239959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 490/980] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:54.240497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2bf8-0035-4270-affd-6c911af6334e None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.240780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2bf8-0035-4270-affd-6c911af6334e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.241051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ee2bf8-0035-4270-affd-6c911af6334e None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.242444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 491/981] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:54.244830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2bf8-0035-4270-affd-6c911af6334e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.257489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81005c-76a7-4b2a-8d88-23408fba95ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.258274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81005c-76a7-4b2a-8d88-23408fba95ed None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.258558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81005c-76a7-4b2a-8d88-23408fba95ed None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.258835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81005c-76a7-4b2a-8d88-23408fba95ed None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.259551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81005c-76a7-4b2a-8d88-23408fba95ed None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.260930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81005c-76a7-4b2a-8d88-23408fba95ed None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-8a81005c-76a7-4b2a-8d88-23408fba95ed None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:54.273157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:54.274331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:54.275636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 491/982] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:00:54.276702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1a11-c69d-47ee-b162-931f674a91f3 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.276702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1a11-c69d-47ee-b162-931f674a91f3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.276702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110e1a11-c69d-47ee-b162-931f674a91f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.292723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-110e1a11-c69d-47ee-b162-931f674a91f3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:54.293933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-110e1a11-c69d-47ee-b162-931f674a91f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:54.350242 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ee2bf8-0035-4270-affd-6c911af6334e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.359508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 492/983] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:54 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:54.360320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1a11-c69d-47ee-b162-931f674a91f3 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.392821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.393486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.397231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.397231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.397231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:54.397231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.397231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.410965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 492/984] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:54.411838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.460219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:54.465808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:54.468754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110e1a11-c69d-47ee-b162-931f674a91f3 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.502481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.503157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.503470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.503747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.504466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:54.504781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.506184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.510885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 493/985] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:54.511286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96536f6-688d-49b0-be5b-656a55b30573 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.511681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.546853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.547599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.547883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.548145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.548809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.550178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:54.561935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:54.562951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:54.563850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 493/986] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:54.564691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.614662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4be026f-6abe-476e-b5fb-5efa3876eb21 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.648337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.649130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.649411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.649684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.656365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.657814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.662270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712b81ac-564f-4450-b7f7-5d25585fa728 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:00:54.671728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:54.672742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 494/987] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:54.673654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.677751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:54.698555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:54.699627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.700313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.700594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.700847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.701555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:54.701866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.703276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.708089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 494/988] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:54.708852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.771609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a320607b-9a56-47fd-867b-a5a9eb1bb2ab None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.805087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.805692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd8b12-48ac-45bb-b388-cc932c70614e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.806131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.806131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.806272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.806880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] RBAC: Adding query filter params (name=compute) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:54.807191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.808613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.813264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 495/989] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:54.814026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.838138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.838792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.839074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.839381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.840172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:54.840474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.841913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.854267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 495/990] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:54.855039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.893668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:54.907613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:54.921346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e11f7f0-7104-4745-85f5-190f4e4782be None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.954740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.955451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.955741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.956018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.956831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] RBAC: Adding query filter params (interface=admin, service_id=7eaee7644d25417ab30bf5fe26d74915, region_id=RegionOne) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:54.957145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.958534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:54.959056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eee67a-a657-4b33-9812-a12a10913981 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:54.963702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 496/991] 10.222.0.69 () {56 vars in 1249 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/endpoints?service_id=7eaee7644d25417ab30bf5fe26d74915&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:54.964479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:54.994479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:54.995148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:54.995467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:54.995741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:54.996451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:54.996757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:54.998206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:55.003187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 496/992] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:55.004135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:55.066806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d8e5ff-bec9-4817-83d8-e41863d36672 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:55.100583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:55.100780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:55.101291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:55.101577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:55.101856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:55.102979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] RBAC: Adding query filter params (role.id=8bdf6fdbab194849a9ac0170382b070b, scope.project.id=569782752acb4085ad742330af999bc2, user.id=6cd71c35c4224658aa2ab0b81f0f4b97) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:55.103401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:55.104142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5efc12-d0bf-43d5-9fc5-86b4c42fbef8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:55.104892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:55.111026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 497/993] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:00:54 2022] GET /identity/v3/role_assignments?user.id=6cd71c35c4224658aa2ab0b81f0f4b97&scope.project.id=569782752acb4085ad742330af999bc2&role.id=8bdf6fdbab194849a9ac0170382b070b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:55.111850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:55.132212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:55.142861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e47b97-21c1-4c21-96b3-6bf1ec1c9d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:55.143579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e47b97-21c1-4c21-96b3-6bf1ec1c9d1e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:55.143860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e47b97-21c1-4c21-96b3-6bf1ec1c9d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:55.144128 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e47b97-21c1-4c21-96b3-6bf1ec1c9d1e None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:55.144885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e47b97-21c1-4c21-96b3-6bf1ec1c9d1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:55.146289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e47b97-21c1-4c21-96b3-6bf1ec1c9d1e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:55.162825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 497/994] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:00:55 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:00:55.231602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999b17c-5cd8-48ca-b46f-da96895009a5 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:55.273845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8dde85-1736-4986-8655-45eae53befb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:55.274532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dde85-1736-4986-8655-45eae53befb6 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:55.274810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dde85-1736-4986-8655-45eae53befb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:55.275084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8dde85-1736-4986-8655-45eae53befb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:55.275939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8dde85-1736-4986-8655-45eae53befb6 None admin] RBAC: Adding query filter params (interface=public, service_id=79b3493abed04d34b2e291c27dcdfab5, region_id=RegionOne) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:55.276247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8dde85-1736-4986-8655-45eae53befb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:55.277667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8dde85-1736-4986-8655-45eae53befb6 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:55.283944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 498/995] 10.222.0.69 () {56 vars in 1251 bytes} [Fri May 20 09:00:55 2022] GET /identity/v3/endpoints?service_id=79b3493abed04d34b2e291c27dcdfab5&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:55.290341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e47b97-21c1-4c21-96b3-6bf1ec1c9d1e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:55.402503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e47b97-21c1-4c21-96b3-6bf1ec1c9d1e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:55.436532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda6c84e-9986-444d-b987-96141a4c3f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:55.437191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6c84e-9986-444d-b987-96141a4c3f6f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:55.437468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6c84e-9986-444d-b987-96141a4c3f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:55.437738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda6c84e-9986-444d-b987-96141a4c3f6f None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:55.438420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6c84e-9986-444d-b987-96141a4c3f6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:55.439817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda6c84e-9986-444d-b987-96141a4c3f6f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:55.452377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 498/996] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:00:55 2022] GET /identity/v3/services => generated 1727 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:00:57.356012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ada7cb-0c4e-49f2-9031-743116060389 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.356565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ada7cb-0c4e-49f2-9031-743116060389 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.356565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ada7cb-0c4e-49f2-9031-743116060389 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.359300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 499/997] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:57.361287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c587-3ea7-4b7c-a5a2-3ac8dc3c8d17 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.361678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c587-3ea7-4b7c-a5a2-3ac8dc3c8d17 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.361878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3c587-3ea7-4b7c-a5a2-3ac8dc3c8d17 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.363304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6951e-5a7f-475d-9177-ce142e9587ea None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.365336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6951e-5a7f-475d-9177-ce142e9587ea None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.365470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 499/998] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:57.365587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a6951e-5a7f-475d-9177-ce142e9587ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.368394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b868a0e-e6c9-4c9c-a26a-9d92bb4811e3 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.368687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b868a0e-e6c9-4c9c-a26a-9d92bb4811e3 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.368964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b868a0e-e6c9-4c9c-a26a-9d92bb4811e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.383466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-80a6951e-5a7f-475d-9177-ce142e9587ea None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:57.384615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-80a6951e-5a7f-475d-9177-ce142e9587ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:57.385484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3b868a0e-e6c9-4c9c-a26a-9d92bb4811e3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:57.386671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3b868a0e-e6c9-4c9c-a26a-9d92bb4811e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:57.411359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:57.454973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 500/999] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:57.458217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 500/1000] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:00:57.479111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6951e-5a7f-475d-9177-ce142e9587ea None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:57.480437 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b868a0e-e6c9-4c9c-a26a-9d92bb4811e3 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:57.492025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:57.588991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b868a0e-e6c9-4c9c-a26a-9d92bb4811e3 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:57.589433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a6951e-5a7f-475d-9177-ce142e9587ea None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:57.625413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:57.625812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:57.626123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.626405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.626626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.626722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] PATH_INFO: `/v3/services/image` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.626913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.627186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.627387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:57.627879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:57.628730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:57.629288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:57.640665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:57.641639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:00:57.642481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:57.643223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:57.644525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 20 09:00:57.645330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:00:57.646055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 501/1001] 10.222.0.69 () {56 vars in 1105 bytes} [Fri May 20 09:00:57 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:57.651475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 501/1002] 10.222.0.69 () {56 vars in 1126 bytes} [Fri May 20 09:00:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:57.651620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:57.654066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:57.673667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:57.726910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:57.748074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019dccd8-e4ee-4fa8-9151-743ac0bffa67 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:57.751409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c7a798-bbdf-4d89-b84e-3031279cbca0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:57.784776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ea522-3c68-4d69-b98e-08f9ec185476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:57.785192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:57.785443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea522-3c68-4d69-b98e-08f9ec185476 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.785719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea522-3c68-4d69-b98e-08f9ec185476 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.785814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.785990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea522-3c68-4d69-b98e-08f9ec185476 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.786083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.786372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.786705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ea522-3c68-4d69-b98e-08f9ec185476 None admin] RBAC: Adding query filter params (name=image) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:57.787010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ea522-3c68-4d69-b98e-08f9ec185476 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:57.787211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:57.787482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:57.788502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ea522-3c68-4d69-b98e-08f9ec185476 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:57.789214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:57.793865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 502/1003] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:57.794437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 502/1004] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:00:57.795076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1491518-cbdc-4ff9-ba6b-eb97935b2177 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.800124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1491518-cbdc-4ff9-ba6b-eb97935b2177 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.800124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1491518-cbdc-4ff9-ba6b-eb97935b2177 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.800124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 503/1005] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:00:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:00:57.801153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:57.801621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1491518-cbdc-4ff9-ba6b-eb97935b2177 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:57.885373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:57.900784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1491518-cbdc-4ff9-ba6b-eb97935b2177 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:57.901031 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aebafa-9835-47c3-b9b0-2f59e4ebc057 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:57.934051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:57.934249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7420385-b058-4ffe-8f1d-6a45ec3450ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:57.934709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.934808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7420385-b058-4ffe-8f1d-6a45ec3450ab None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.934986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.935090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7420385-b058-4ffe-8f1d-6a45ec3450ab None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.935253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.935380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7420385-b058-4ffe-8f1d-6a45ec3450ab None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.935981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7420385-b058-4ffe-8f1d-6a45ec3450ab None admin] RBAC: Adding query filter params (type=image) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:57.936072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:57.936282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:57.936373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7420385-b058-4ffe-8f1d-6a45ec3450ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:57.937648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:57.937748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7420385-b058-4ffe-8f1d-6a45ec3450ab None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:57.942266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 504/1006] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:00:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:57.947550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 503/1007] 10.222.0.69 () {56 vars in 1122 bytes} [Fri May 20 09:00:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:57.947758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355a335-8422-4b9b-8d72-508fc6c9cdfd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:57.947758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355a335-8422-4b9b-8d72-508fc6c9cdfd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:57.947758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8355a335-8422-4b9b-8d72-508fc6c9cdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:57.948490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:57.956633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:57.962060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8355a335-8422-4b9b-8d72-508fc6c9cdfd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:00:57.963528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8355a335-8422-4b9b-8d72-508fc6c9cdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:00:58.042314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 505/1008] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:00:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:00:58.059391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355a335-8422-4b9b-8d72-508fc6c9cdfd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:58.070796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c8b548-e825-48e9-a1ab-d07792be8947 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:58.115817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a07725-c0c2-4846-a89a-8364b80f3f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:58.116512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07725-c0c2-4846-a89a-8364b80f3f0f None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:58.116953 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07725-c0c2-4846-a89a-8364b80f3f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:58.117357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a07725-c0c2-4846-a89a-8364b80f3f0f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:58.118121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a07725-c0c2-4846-a89a-8364b80f3f0f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:58.120139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a07725-c0c2-4846-a89a-8364b80f3f0f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:58.140130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 504/1009] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:00:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:00:58.149369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:58.177710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8355a335-8422-4b9b-8d72-508fc6c9cdfd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:58.221580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:58.222383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:58.222662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:58.222939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] PATH_INFO: `/v3/services/compute` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:58.223622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:58.225079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:00:58.237727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:00:58.238897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:00:58.240294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 506/1010] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:00:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:00:58.245220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a07725-c0c2-4846-a89a-8364b80f3f0f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:58.311767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:58.356675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a07725-c0c2-4846-a89a-8364b80f3f0f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:58.391192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:58.391961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:58.392351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:58.392717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:58.393527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:58.393926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:58.395410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:58.405059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 505/1011] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:58.406533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:58.512452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564d987-232f-4170-b945-f5ab5ac8f1ff None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:58.553709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18261bf6-4a68-44de-b9dc-8e5da3cb4d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:58.554375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18261bf6-4a68-44de-b9dc-8e5da3cb4d40 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:58.554652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18261bf6-4a68-44de-b9dc-8e5da3cb4d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:58.554909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18261bf6-4a68-44de-b9dc-8e5da3cb4d40 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:58.555650 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18261bf6-4a68-44de-b9dc-8e5da3cb4d40 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:00:58.555952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18261bf6-4a68-44de-b9dc-8e5da3cb4d40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:58.557321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18261bf6-4a68-44de-b9dc-8e5da3cb4d40 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:58.562550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 507/1012] 10.222.0.69 () {56 vars in 1112 bytes} [Fri May 20 09:00:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:00:58.568089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:00:58.685499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392ada41-18bc-4f96-ae0a-fc485d29ab3b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:00:58.711725 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:00:58.722342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c695d3-d08d-4754-9bbc-5d946f8994d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:00:58.723205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c695d3-d08d-4754-9bbc-5d946f8994d1 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:00:58.723585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c695d3-d08d-4754-9bbc-5d946f8994d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:00:58.723923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c695d3-d08d-4754-9bbc-5d946f8994d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:00:58.724742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c695d3-d08d-4754-9bbc-5d946f8994d1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:00:58.726239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c695d3-d08d-4754-9bbc-5d946f8994d1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:00:58.751282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 506/1013] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:00:58 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:00.156338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08ef6-0fb5-4e54-bad8-da039586a029 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:00.156835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08ef6-0fb5-4e54-bad8-da039586a029 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:00.156835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd08ef6-0fb5-4e54-bad8-da039586a029 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:00.158235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 508/1014] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:00.163718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde11b-df7c-4c39-a2c4-251cd07e3a15 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:00.164515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde11b-df7c-4c39-a2c4-251cd07e3a15 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:00.164604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bde11b-df7c-4c39-a2c4-251cd07e3a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:00.181530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-56bde11b-df7c-4c39-a2c4-251cd07e3a15 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:00.182707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-56bde11b-df7c-4c39-a2c4-251cd07e3a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:00.261624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 507/1015] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:00 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:00.283781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd08ef6-0fb5-4e54-bad8-da039586a029 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:00.287882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:00.388405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9253-fb6a-4d01-b543-4227b25219f1 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:00.388620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd08ef6-0fb5-4e54-bad8-da039586a029 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:00.388733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9253-fb6a-4d01-b543-4227b25219f1 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:00.389022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ec9253-fb6a-4d01-b543-4227b25219f1 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:00.390502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 508/1016] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:00.395531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7f8e5-5e20-44f0-bbeb-9968bc8d6f74 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:00.395827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7f8e5-5e20-44f0-bbeb-9968bc8d6f74 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:00.396070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf7f8e5-5e20-44f0-bbeb-9968bc8d6f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:00.413486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-daf7f8e5-5e20-44f0-bbeb-9968bc8d6f74 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:00.414937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-daf7f8e5-5e20-44f0-bbeb-9968bc8d6f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:00.423751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386f900-786e-46f5-8cca-6d0c6abf0e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:00.424399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386f900-786e-46f5-8cca-6d0c6abf0e85 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:00.424690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386f900-786e-46f5-8cca-6d0c6abf0e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:00.424961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386f900-786e-46f5-8cca-6d0c6abf0e85 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:00.425675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386f900-786e-46f5-8cca-6d0c6abf0e85 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:00.427053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386f900-786e-46f5-8cca-6d0c6abf0e85 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:00.447126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 509/1017] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:01:00 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:00.511573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 509/1018] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:00 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:01:00.534457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386f900-786e-46f5-8cca-6d0c6abf0e85 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:00.604386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:00.651798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386f900-786e-46f5-8cca-6d0c6abf0e85 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:00.686659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0cd74-1d1b-466a-b062-7b8e56e3ade4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:00.687687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0cd74-1d1b-466a-b062-7b8e56e3ade4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:00.688035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0cd74-1d1b-466a-b062-7b8e56e3ade4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:00.688304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0cd74-1d1b-466a-b062-7b8e56e3ade4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:00.689707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0cd74-1d1b-466a-b062-7b8e56e3ade4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:00.691232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0cd74-1d1b-466a-b062-7b8e56e3ade4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:00.695307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 510/1019] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:00.698992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7f8e5-5e20-44f0-bbeb-9968bc8d6f74 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:00.732785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:00.804966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf7f8e5-5e20-44f0-bbeb-9968bc8d6f74 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:00.839939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc33cdf-d820-4c86-a7bc-2d35b9e7cd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:00.841014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc33cdf-d820-4c86-a7bc-2d35b9e7cd01 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:00.841514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc33cdf-d820-4c86-a7bc-2d35b9e7cd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:00.841980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc33cdf-d820-4c86-a7bc-2d35b9e7cd01 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:00.842907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc33cdf-d820-4c86-a7bc-2d35b9e7cd01 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:00.844582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc33cdf-d820-4c86-a7bc-2d35b9e7cd01 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:00.873271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 510/1020] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:01:00 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:01:01.181627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4506b-9784-4fa5-934b-42a313be2476 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:01.182197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4506b-9784-4fa5-934b-42a313be2476 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:01.182197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4506b-9784-4fa5-934b-42a313be2476 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:01.183576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 511/1021] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:01.189171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4a7eb-becf-408e-88c1-cc455f60948d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:01.189587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4a7eb-becf-408e-88c1-cc455f60948d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:01.189771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4a7eb-becf-408e-88c1-cc455f60948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:01.208043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3ab4a7eb-becf-408e-88c1-cc455f60948d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:01.209295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3ab4a7eb-becf-408e-88c1-cc455f60948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:01.281695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 511/1022] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:01 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:01.304977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4506b-9784-4fa5-934b-42a313be2476 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:01.407621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4506b-9784-4fa5-934b-42a313be2476 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:01.436926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:01.443049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f654d-a4ec-4244-bbdf-7dfc774e350f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:01.443921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f654d-a4ec-4244-bbdf-7dfc774e350f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:01.444202 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f654d-a4ec-4244-bbdf-7dfc774e350f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:01.444478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f654d-a4ec-4244-bbdf-7dfc774e350f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:01.454976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f654d-a4ec-4244-bbdf-7dfc774e350f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-dc8f654d-a4ec-4244-bbdf-7dfc774e350f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:01.463033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:01.464137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:01.465001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 512/1023] 10.222.0.69 () {56 vars in 1120 bytes} [Fri May 20 09:01:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:01:01.472054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4a7eb-becf-408e-88c1-cc455f60948d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:01.491042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:01.579662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4a7eb-becf-408e-88c1-cc455f60948d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:01.617672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:01.618351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:01.618632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:01.618904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:01.619699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:01.620006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:01.621392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:01.629810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 512/1024] 10.222.0.69 () {56 vars in 1115 bytes} [Fri May 20 09:01:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:01.635439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f654d-a4ec-4244-bbdf-7dfc774e350f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:01.745737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f654d-a4ec-4244-bbdf-7dfc774e350f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:01.786644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:01.787856 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:01.788375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:01.789377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:01.800950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:01.802653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:01:01.820902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:01.821842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:01.822796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:01:01.822796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:01:01.822796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:01.822796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:01.824929 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 513/1025] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:01.830536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:01.911334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:01.933496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905add9-ecce-45d0-aafa-21525f0fdd3f None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:01.967898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:01.968558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:01.968833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:01.969103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:01.969822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:01.970127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:01.971537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:01.983464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 513/1026] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:01.988854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:02.035100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:02.091119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebee51a-b89d-46b6-94e7-1f0b3b510986 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:02.124337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:02.125130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:02.125407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:02.125675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:02.133595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:02.135033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:01:02.148904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:02.149920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:01:02.150877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:02.154936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 514/1027] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:01:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:02.156243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:02.260217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef4ca75-dd59-4958-b5b8-94d17ae621ca None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:02.296019 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:02.296614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:02.296895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:02.297148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:02.298020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:02.298329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:02.299810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:02.308384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:02.323308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 514/1028] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:01:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:02.325430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:02.429062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72443ca-142f-42cc-b9ca-ecb949acf6c4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:02.431448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:02.465352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e180c-08fe-4149-8be2-3f9a85069b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:02.465997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e180c-08fe-4149-8be2-3f9a85069b50 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:02.466271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e180c-08fe-4149-8be2-3f9a85069b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:02.466540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e180c-08fe-4149-8be2-3f9a85069b50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:02.537917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e180c-08fe-4149-8be2-3f9a85069b50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:02.539773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e180c-08fe-4149-8be2-3f9a85069b50 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:02.641647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 515/1029] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:02 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:02.647188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:02.679520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7c48-060e-4d9d-aa9f-f355b65ddf92 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:02.679812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7c48-060e-4d9d-aa9f-f355b65ddf92 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:02.680072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbd7c48-060e-4d9d-aa9f-f355b65ddf92 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:02.681834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 516/1030] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:01:02.686658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fa90e-7ae2-460e-a1e0-2b507c4b27ee None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:02.686936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fa90e-7ae2-460e-a1e0-2b507c4b27ee None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:02.687212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1fa90e-7ae2-460e-a1e0-2b507c4b27ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:02.693238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:02.705786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-ab1fa90e-7ae2-460e-a1e0-2b507c4b27ee None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:02.706982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ab1fa90e-7ae2-460e-a1e0-2b507c4b27ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:02.749646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a462f0d1-288c-455b-894e-217e3ec98286 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:02.785867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 517/1031] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:01:02.791722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:02.792563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:02.792842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:02.793120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:02.803742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:02.805201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:02.808074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fa90e-7ae2-460e-a1e0-2b507c4b27ee None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:01:02.824621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:02.825515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:02.826410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:01:02.826410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:01:02.826410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:02.826410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:02.826410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 515/1032] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:02.831507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:02.883429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:02.910983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1fa90e-7ae2-460e-a1e0-2b507c4b27ee None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:02.942319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4670d43-5aaf-43ff-b804-57bdc2279628 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:02.946833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:02.947673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:02.947957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:02.948229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] PATH_INFO: `/v3/services/image` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:02.948928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:02.950351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:02.952739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:02.965077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:02.966182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:01:02.967117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:02.971060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 518/1033] 10.222.0.69 () {56 vars in 1105 bytes} [Fri May 20 09:01:02 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:01:02.977721 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:02.984557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:02.985244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:02.985568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:02.985851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:02.986589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:02.986895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:02.988329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.001368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 516/1034] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:03.006408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:03.085851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a28934-ab68-4ca5-99bd-098324de5d51 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.113561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74464af8-1da9-4c73-bc14-10b2912dbe07 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.120186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.120864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.120864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.121145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.121788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] RBAC: Adding query filter params (name=image) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:03.122091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.123560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.125052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:03.129208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 519/1035] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:02 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:03.135195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:03.153425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def0fdb-6f34-47ad-b16f-bd7f34c79cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.154230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def0fdb-6f34-47ad-b16f-bd7f34c79cfa None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.154508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def0fdb-6f34-47ad-b16f-bd7f34c79cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.154776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def0fdb-6f34-47ad-b16f-bd7f34c79cfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.161407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def0fdb-6f34-47ad-b16f-bd7f34c79cfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.162878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def0fdb-6f34-47ad-b16f-bd7f34c79cfa None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-1def0fdb-6f34-47ad-b16f-bd7f34c79cfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:01:03.176884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:03.177757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 517/1036] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756a23-2a62-4921-b838-84d0a20b367b None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.183537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756a23-2a62-4921-b838-84d0a20b367b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.184155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb756a23-2a62-4921-b838-84d0a20b367b None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.184155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 518/1037] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:03.184564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb756a23-2a62-4921-b838-84d0a20b367b None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:03.196934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:03.235156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e619959-d02b-41ff-b7e9-8a247035cf43 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.267648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f7fee-9315-489a-b941-4a1e9050e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.268302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f7fee-9315-489a-b941-4a1e9050e3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.268581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f7fee-9315-489a-b941-4a1e9050e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.268854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f7fee-9315-489a-b941-4a1e9050e3c7 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.269559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f7fee-9315-489a-b941-4a1e9050e3c7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:03.269859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f7fee-9315-489a-b941-4a1e9050e3c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.271238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f7fee-9315-489a-b941-4a1e9050e3c7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.276359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 520/1038] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:03.277599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dadaf-9e7e-403e-a4a1-7220bfa8c8cd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.277883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dadaf-9e7e-403e-a4a1-7220bfa8c8cd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.278145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dadaf-9e7e-403e-a4a1-7220bfa8c8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.288443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb756a23-2a62-4921-b838-84d0a20b367b None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.295955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e81dadaf-9e7e-403e-a4a1-7220bfa8c8cd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:03.297126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e81dadaf-9e7e-403e-a4a1-7220bfa8c8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:03.322520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.323181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.323488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.323764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.324560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:03.324864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.326305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.338105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 519/1039] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:01:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:03.338865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:03.363118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 521/1040] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:03 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:03.363980 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dadaf-9e7e-403e-a4a1-7220bfa8c8cd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:03.371805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:03.419418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:03.445094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b70b9a-6120-442f-b7e4-2f5115d2ebe4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.460749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dadaf-9e7e-403e-a4a1-7220bfa8c8cd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.477873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.478529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.478813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.479085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.479911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] RBAC: Adding query filter params (interface=public, service_id=72f3c968b028426284e7dcc58f5ec4e7, region_id=RegionOne) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:03.480196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.481577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.486591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 520/1041] 10.222.0.69 () {56 vars in 1251 bytes} [Fri May 20 09:01:03 2022] GET /identity/v3/endpoints?service_id=72f3c968b028426284e7dcc58f5ec4e7&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:03.487384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:03.495093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.495770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.496115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.496391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.497214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] RBAC: Adding query filter params (role.id=8bdf6fdbab194849a9ac0170382b070b, scope.project.id=569782752acb4085ad742330af999bc2, user.id=04cd27bb637a404d827d384196eabfb1) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:03.497516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.499007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.515289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 522/1042] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:03 2022] GET /identity/v3/role_assignments?user.id=04cd27bb637a404d827d384196eabfb1&scope.project.id=569782752acb4085ad742330af999bc2&role.id=8bdf6fdbab194849a9ac0170382b070b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:03.515289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:03.586268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:03.595644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720346d9-a221-4686-b512-ed6f25eb9e1e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.629914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bb298e-6460-4b43-b39d-9578e7da42e5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.630853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d4667-9b18-494c-8318-a87b4d25708e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.631729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d4667-9b18-494c-8318-a87b4d25708e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.632012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d4667-9b18-494c-8318-a87b4d25708e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.632290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d4667-9b18-494c-8318-a87b4d25708e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.633510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d4667-9b18-494c-8318-a87b4d25708e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.634996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d4667-9b18-494c-8318-a87b4d25708e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.636736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 521/1043] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:03.642591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d4667-9b18-494c-8318-a87b4d25708e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:03.654956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:03.672760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e32a842-b845-4ea6-8929-ac9202429111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.673308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32a842-b845-4ea6-8929-ac9202429111 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.673589 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32a842-b845-4ea6-8929-ac9202429111 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.673891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e32a842-b845-4ea6-8929-ac9202429111 None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.674577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32a842-b845-4ea6-8929-ac9202429111 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.676031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e32a842-b845-4ea6-8929-ac9202429111 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.691290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 523/1044] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:01:03 2022] GET /identity/v3/services => generated 1957 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:01:03.759907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d4667-9b18-494c-8318-a87b4d25708e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:03.805854 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4d9e4f-9bab-4f38-866b-448d14783fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:03.806564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d9e4f-9bab-4f38-866b-448d14783fae None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:03.806847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d9e4f-9bab-4f38-866b-448d14783fae None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:03.807096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4d9e4f-9bab-4f38-866b-448d14783fae None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:03.807871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d9e4f-9bab-4f38-866b-448d14783fae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:03.809349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4d9e4f-9bab-4f38-866b-448d14783fae None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:03.839303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 522/1045] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 09:01:03 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:01:05.705014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744705fd-fa94-43fc-af0e-d12e4a7048fb None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:05.705586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744705fd-fa94-43fc-af0e-d12e4a7048fb None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:05.705586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744705fd-fa94-43fc-af0e-d12e4a7048fb None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:05.707246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 524/1046] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:05.713115 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f102f-058c-4bd4-ac9a-b0778074e15d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:05.713683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f102f-058c-4bd4-ac9a-b0778074e15d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:05.713683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f102f-058c-4bd4-ac9a-b0778074e15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:05.732918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-7d1f102f-058c-4bd4-ac9a-b0778074e15d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:05.734182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7d1f102f-058c-4bd4-ac9a-b0778074e15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:05.746939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:05.812786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 523/1047] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:05.836357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744705fd-fa94-43fc-af0e-d12e4a7048fb None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:05.901559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf17670-4385-4311-bbc6-75deca69afdd None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:05.902182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf17670-4385-4311-bbc6-75deca69afdd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:05.902469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf17670-4385-4311-bbc6-75deca69afdd None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:05.904523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 524/1048] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:05.910246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd80775-c892-47db-a5d4-8b0c10eabdf4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:05.910543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd80775-c892-47db-a5d4-8b0c10eabdf4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:05.910809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd80775-c892-47db-a5d4-8b0c10eabdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:05.928855 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-fbd80775-c892-47db-a5d4-8b0c10eabdf4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:05.930299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fbd80775-c892-47db-a5d4-8b0c10eabdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:05.961477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744705fd-fa94-43fc-af0e-d12e4a7048fb None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:05.997951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f00f-31fa-401e-9414-313e68004218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:05.998836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f00f-31fa-401e-9414-313e68004218 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:05.999130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f00f-31fa-401e-9414-313e68004218 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:05.999439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e024f00f-31fa-401e-9414-313e68004218 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.003439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 525/1049] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:06.012307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e024f00f-31fa-401e-9414-313e68004218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-e024f00f-31fa-401e-9414-313e68004218 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:06.020730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:06.021858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:06.028321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 525/1050] 10.222.0.69 () {56 vars in 1120 bytes} [Fri May 20 09:01:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:06.029059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd80775-c892-47db-a5d4-8b0c10eabdf4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.030693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f00f-31fa-401e-9414-313e68004218 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.082414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:06.102351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:06.125673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd80775-c892-47db-a5d4-8b0c10eabdf4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.135478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e024f00f-31fa-401e-9414-313e68004218 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.158861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64daa68-c447-4e3c-8568-cf259f653349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.159690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64daa68-c447-4e3c-8568-cf259f653349 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.159975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64daa68-c447-4e3c-8568-cf259f653349 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.160248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64daa68-c447-4e3c-8568-cf259f653349 None admin] PATH_INFO: `/v3/services/image` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.160907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64daa68-c447-4e3c-8568-cf259f653349 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.162353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64daa68-c447-4e3c-8568-cf259f653349 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.169551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349b85e3-c254-4603-9483-34de257b82f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.170209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b85e3-c254-4603-9483-34de257b82f1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.170487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b85e3-c254-4603-9483-34de257b82f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.170758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349b85e3-c254-4603-9483-34de257b82f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.171528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b85e3-c254-4603-9483-34de257b82f1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:06.171832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b85e3-c254-4603-9483-34de257b82f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.173234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349b85e3-c254-4603-9483-34de257b82f1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-a64daa68-c447-4e3c-8568-cf259f653349 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:06.174153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:06.175099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:06.175984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 526/1051] 10.222.0.69 () {56 vars in 1105 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/services/image => generated 86 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:06.176786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9781ae-da5a-486f-9053-a4a1091acd23 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.176965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9781ae-da5a-486f-9053-a4a1091acd23 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.177504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9781ae-da5a-486f-9053-a4a1091acd23 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.178559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 527/1052] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:06.183189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9781ae-da5a-486f-9053-a4a1091acd23 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.186224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 526/1053] 10.222.0.69 () {56 vars in 1115 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:06.187524 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb95096-c486-4c9a-bcc1-c0d98e1be0c6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.187810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb95096-c486-4c9a-bcc1-c0d98e1be0c6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.188080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb95096-c486-4c9a-bcc1-c0d98e1be0c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.204863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5bb95096-c486-4c9a-bcc1-c0d98e1be0c6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:06.206020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5bb95096-c486-4c9a-bcc1-c0d98e1be0c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:06.270791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 527/1054] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:06.271907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb95096-c486-4c9a-bcc1-c0d98e1be0c6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.284145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9781ae-da5a-486f-9053-a4a1091acd23 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.316364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:06.319954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.320625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.320912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.321194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.321911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] RBAC: Adding query filter params (name=image) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:06.322221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.323653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.328818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 528/1055] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:06.329526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.333508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:06.380354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb95096-c486-4c9a-bcc1-c0d98e1be0c6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.412084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.412911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.413184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.413453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.423559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.425003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.428220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d49660-6d39-44f9-b33d-be258bba2836 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:01:06.441996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:06.443010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:06.444036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:01:06.444036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:01:06.444036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:06.444036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:06.444036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 528/1056] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:06.444036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.470552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575ffc8-8f6a-44e0-b7d8-410a86607dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.471378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ffc8-8f6a-44e0-b7d8-410a86607dfd None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.471639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ffc8-8f6a-44e0-b7d8-410a86607dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.471932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575ffc8-8f6a-44e0-b7d8-410a86607dfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.481939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6575ffc8-8f6a-44e0-b7d8-410a86607dfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-6575ffc8-8f6a-44e0-b7d8-410a86607dfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:06.489427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:06.490681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:06.493446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:06.495564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 529/1057] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:06.496406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575ffc8-8f6a-44e0-b7d8-410a86607dfd None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.531289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:06.543929 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073f873-d81d-4e64-a269-d1579763ee38 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.558959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:06.577070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.577734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.578026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.578301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.579045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] RBAC: Adding query filter params (type=image) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:06.579381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.580764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.585523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 529/1058] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:06.586286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.600793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575ffc8-8f6a-44e0-b7d8-410a86607dfd None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.633422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.634194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.634481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.634762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.635548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:06.635845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.637230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.648888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 530/1059] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:06.649659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.684225 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e910da-fb5b-4fb3-916c-143810bae15b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.717285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.717912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.718205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.718477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.719213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:06.719561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.720921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.729049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 530/1060] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:06.729668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.747463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df660c16-19a2-42c9-8ad8-3078a50e284d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.748240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:06.773687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:06.785748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cb420-8f88-45eb-bcb4-5d77ec2b62b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.786325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cb420-8f88-45eb-bcb4-5d77ec2b62b2 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.786609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cb420-8f88-45eb-bcb4-5d77ec2b62b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.786883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438cb420-8f88-45eb-bcb4-5d77ec2b62b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.787739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cb420-8f88-45eb-bcb4-5d77ec2b62b2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.789459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438cb420-8f88-45eb-bcb4-5d77ec2b62b2 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.808535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 531/1061] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:01:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:06.809667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cb420-8f88-45eb-bcb4-5d77ec2b62b2 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.836921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdf688d-7165-4adf-8100-6951b82f9159 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.882402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.883592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.884045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.884489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.893510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.895341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:01:06.913082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:06.914538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:06.915693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 531/1062] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:06.920956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:06.931079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438cb420-8f88-45eb-bcb4-5d77ec2b62b2 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:06.970337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704b33a-9c25-4f3f-9ceb-072d1ae85c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:06.971642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704b33a-9c25-4f3f-9ceb-072d1ae85c56 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:06.972150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704b33a-9c25-4f3f-9ceb-072d1ae85c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:06.972577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4704b33a-9c25-4f3f-9ceb-072d1ae85c56 None admin] PATH_INFO: `/v3/users/c7a8c94b3b3b46fe938600b73aa75ee2` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:06.989268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704b33a-9c25-4f3f-9ceb-072d1ae85c56 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a8c94b3b3b46fe938600b73aa75ee2)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:06.991075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4704b33a-9c25-4f3f-9ceb-072d1ae85c56 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:06.993376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 532/1063] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/users/c7a8c94b3b3b46fe938600b73aa75ee2 => generated 285 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:06.999993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704b33a-9c25-4f3f-9ceb-072d1ae85c56 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:07.001291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:07.040344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114e282-006e-4995-bd08-f977cf320528 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.044343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:07.075949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.076624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.077060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.077338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.078970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:07.079302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.080732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.098557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 532/1064] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:07.104310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:07.108515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4704b33a-9c25-4f3f-9ceb-072d1ae85c56 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.144384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc70a8-196e-431c-9f50-79e680c5d1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.144806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc70a8-196e-431c-9f50-79e680c5d1e4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.145077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc70a8-196e-431c-9f50-79e680c5d1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.145352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc70a8-196e-431c-9f50-79e680c5d1e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.206440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bb0bb-13e6-4bfa-a412-b228c12dc26c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.223685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc70a8-196e-431c-9f50-79e680c5d1e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.225178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc70a8-196e-431c-9f50-79e680c5d1e4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.232261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:07.241974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.242771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.243049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.243326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.253740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.255182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.266747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:01:07.273445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:07.274388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:07.275366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:01:07.275366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:01:07.275366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:07.275366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:07.275366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 533/1065] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:01:07.280348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:07.334438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 533/1066] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:07.339848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc70a8-196e-431c-9f50-79e680c5d1e4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:07.385249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db30c681-8a13-4b62-ba9d-ded0b712ac7c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.421151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.421812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.422091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.422477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.423175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:07.423582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.424984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.436616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 534/1067] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:07.444450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:07.459677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc70a8-196e-431c-9f50-79e680c5d1e4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.488423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:07.496224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bad44-23fa-4aba-91eb-c8c53e0652be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.497202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bad44-23fa-4aba-91eb-c8c53e0652be None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.497431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bad44-23fa-4aba-91eb-c8c53e0652be None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.497746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6bad44-23fa-4aba-91eb-c8c53e0652be None admin] PATH_INFO: `/v3/projects/ca30252be2274a08a1d81bca2de0ec13` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.499286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bad44-23fa-4aba-91eb-c8c53e0652be None admin] RBAC: Authorizing `identity:get_project(project_id=ca30252be2274a08a1d81bca2de0ec13)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.500785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6bad44-23fa-4aba-91eb-c8c53e0652be None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.503287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 534/1068] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:07 2022] GET /identity/v3/projects/ca30252be2274a08a1d81bca2de0ec13 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:07.508529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bad44-23fa-4aba-91eb-c8c53e0652be None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:07.517730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:07.558503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b01b6-e529-4172-8010-62052f696b31 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.594263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.594972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.595253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.595552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.602761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.604236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.615577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6bad44-23fa-4aba-91eb-c8c53e0652be None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:01:07.618456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:07.619639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:07.620553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 535/1069] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:01:07.625276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:07.654999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.655681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.655959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.656230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.657233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=ca30252be2274a08a1d81bca2de0ec13, user.id=c7a8c94b3b3b46fe938600b73aa75ee2) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:07.657547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.658997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.675290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 535/1070] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:07 2022] GET /identity/v3/role_assignments?user.id=c7a8c94b3b3b46fe938600b73aa75ee2&scope.project.id=ca30252be2274a08a1d81bca2de0ec13&role.id=6dca675ade814980a252f800fb4fa3ad => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:07.735943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:07.740659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b700fcf4-b5c1-4c6c-8df9-fc4acd6c6344 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.786472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3158a3c1-d51c-4903-a236-3b679868fbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.787735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158a3c1-d51c-4903-a236-3b679868fbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.788249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158a3c1-d51c-4903-a236-3b679868fbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.789111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3158a3c1-d51c-4903-a236-3b679868fbb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.790124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158a3c1-d51c-4903-a236-3b679868fbb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:07.790639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158a3c1-d51c-4903-a236-3b679868fbb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.792360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3158a3c1-d51c-4903-a236-3b679868fbb4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.807021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 536/1071] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:01:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:07.812968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:07.885747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:07.918068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb02579-2821-4324-bd64-59674b0c3024 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:07.952561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8241f9-43f3-4bdc-a6f0-83d20cf9bc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:07.953480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8241f9-43f3-4bdc-a6f0-83d20cf9bc2d None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:07.953756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8241f9-43f3-4bdc-a6f0-83d20cf9bc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:07.954030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8241f9-43f3-4bdc-a6f0-83d20cf9bc2d None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/04cd27bb637a404d827d384196eabfb1/roles/8bdf6fdbab194849a9ac0170382b070b` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:07.956413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8241f9-43f3-4bdc-a6f0-83d20cf9bc2d None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=04cd27bb637a404d827d384196eabfb1, role_id=8bdf6fdbab194849a9ac0170382b070b)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:07.957896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8241f9-43f3-4bdc-a6f0-83d20cf9bc2d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:07.972664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 536/1072] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:01:07 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/04cd27bb637a404d827d384196eabfb1/roles/8bdf6fdbab194849a9ac0170382b070b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:01:09.039679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb64f-b43d-470a-a6d2-36e20fb291e4 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:09.040193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb64f-b43d-470a-a6d2-36e20fb291e4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:09.040193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242bb64f-b43d-470a-a6d2-36e20fb291e4 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:09.041457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 537/1073] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:09.046980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2a73e-a245-40fd-9f6c-a0f3c21e14ea None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:09.047426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2a73e-a245-40fd-9f6c-a0f3c21e14ea None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:09.047522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2a73e-a245-40fd-9f6c-a0f3c21e14ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:09.066444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a2e2a73e-a245-40fd-9f6c-a0f3c21e14ea None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:09.067690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a2e2a73e-a245-40fd-9f6c-a0f3c21e14ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:09.135958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 537/1074] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:09.157454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bb64f-b43d-470a-a6d2-36e20fb291e4 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:09.266978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242bb64f-b43d-470a-a6d2-36e20fb291e4 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:09.300921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9237c-16d0-4334-a4c9-acf0d0e3c8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:09.301476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9237c-16d0-4334-a4c9-acf0d0e3c8bd None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:09.301785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9237c-16d0-4334-a4c9-acf0d0e3c8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:09.302049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac9237c-16d0-4334-a4c9-acf0d0e3c8bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:09.329174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:09.369621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9237c-16d0-4334-a4c9-acf0d0e3c8bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:09.371083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac9237c-16d0-4334-a4c9-acf0d0e3c8bd None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:09.470162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 538/1075] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:09.476034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2a73e-a245-40fd-9f6c-a0f3c21e14ea None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:09.535660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:09.578202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2a73e-a245-40fd-9f6c-a0f3c21e14ea None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:09.612239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d759b-318d-4f71-ac4e-36f8edf8e82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:09.613045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d759b-318d-4f71-ac4e-36f8edf8e82b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:09.613327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d759b-318d-4f71-ac4e-36f8edf8e82b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:09.613605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d759b-318d-4f71-ac4e-36f8edf8e82b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:09.623813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d759b-318d-4f71-ac4e-36f8edf8e82b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:09.625293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d759b-318d-4f71-ac4e-36f8edf8e82b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-2f8d759b-318d-4f71-ac4e-36f8edf8e82b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:01:09.643375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:09.644296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:09.645220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:01:09.645220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:01:09.645220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:09.645220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:09.647629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 538/1076] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:09.650656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9237c-16d0-4334-a4c9-acf0d0e3c8bd None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:09.719867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:09.768452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac9237c-16d0-4334-a4c9-acf0d0e3c8bd None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:09.805274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a44021-8d73-4b0f-b7ad-ec8b96cb89d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:09.805939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a44021-8d73-4b0f-b7ad-ec8b96cb89d9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:09.806220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a44021-8d73-4b0f-b7ad-ec8b96cb89d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:09.806493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a44021-8d73-4b0f-b7ad-ec8b96cb89d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:09.807219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a44021-8d73-4b0f-b7ad-ec8b96cb89d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:09.807550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a44021-8d73-4b0f-b7ad-ec8b96cb89d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:09.808918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a44021-8d73-4b0f-b7ad-ec8b96cb89d9 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:09.820838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 539/1077] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:09.869329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b7074-2325-4035-ab15-d7715288d44b None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:09.869540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b7074-2325-4035-ab15-d7715288d44b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:09.869828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452b7074-2325-4035-ab15-d7715288d44b None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:09.871296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 539/1078] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:09.877169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a89763-aa3c-4d60-a540-82b9972add10 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:09.877413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a89763-aa3c-4d60-a540-82b9972add10 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:09.877710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a89763-aa3c-4d60-a540-82b9972add10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:09.896398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-18a89763-aa3c-4d60-a540-82b9972add10 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:09.897638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-18a89763-aa3c-4d60-a540-82b9972add10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:09.974220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 540/1079] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:09.997650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b7074-2325-4035-ab15-d7715288d44b None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.113803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452b7074-2325-4035-ab15-d7715288d44b None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.128534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:10.152225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf8eb2-ae45-4a11-b7ec-410a9f60b7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.153272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf8eb2-ae45-4a11-b7ec-410a9f60b7be None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.153735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf8eb2-ae45-4a11-b7ec-410a9f60b7be None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.154005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf8eb2-ae45-4a11-b7ec-410a9f60b7be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.163531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54647b8c-71db-4276-9ea5-3ed22368718d None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.164359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54647b8c-71db-4276-9ea5-3ed22368718d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.164435 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54647b8c-71db-4276-9ea5-3ed22368718d None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.165805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 541/1080] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:01:10.166613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bf8eb2-ae45-4a11-b7ec-410a9f60b7be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:10.171026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58ea40-fbda-4b3e-8b61-288373210afc None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.171343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58ea40-fbda-4b3e-8b61-288373210afc None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.171625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d58ea40-fbda-4b3e-8b61-288373210afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-e5bf8eb2-ae45-4a11-b7ec-410a9f60b7be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:10.174227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:10.175520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:10.176396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 540/1081] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:01:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:10.181331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf8eb2-ae45-4a11-b7ec-410a9f60b7be None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.188432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2d58ea40-fbda-4b3e-8b61-288373210afc None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:10.189607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2d58ea40-fbda-4b3e-8b61-288373210afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:10.234232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:10.255846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 542/1082] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:01:10.279157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58ea40-fbda-4b3e-8b61-288373210afc None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.280474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf8eb2-ae45-4a11-b7ec-410a9f60b7be None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.316479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.317140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.317420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.317685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.318424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:10.318744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:10.320154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:10.328441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 541/1083] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:01:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:10.334318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.367480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:10.377069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d58ea40-fbda-4b3e-8b61-288373210afc None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.410413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7e203-46fc-4966-8f41-4d90756d9bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.411149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7e203-46fc-4966-8f41-4d90756d9bfe None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.411460 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7e203-46fc-4966-8f41-4d90756d9bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.411732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab7e203-46fc-4966-8f41-4d90756d9bfe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.422928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab7e203-46fc-4966-8f41-4d90756d9bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-3ab7e203-46fc-4966-8f41-4d90756d9bfe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:10.431039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:10.432003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 20 09:01:10.432883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:10.433729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72ff55d-40f0-456b-bdf2-7fb5a8950d3d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.437383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 543/1084] 10.222.0.69 () {56 vars in 1120 bytes} [Fri May 20 09:01:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:01:10.438188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7e203-46fc-4966-8f41-4d90756d9bfe None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.465826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0443e-b28f-4160-a836-3a5d21eadd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.466692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0443e-b28f-4160-a836-3a5d21eadd6b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.466967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0443e-b28f-4160-a836-3a5d21eadd6b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.467221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0443e-b28f-4160-a836-3a5d21eadd6b None admin] PATH_INFO: `/v3/users/c7a8c94b3b3b46fe938600b73aa75ee2` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.468400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0443e-b28f-4160-a836-3a5d21eadd6b None admin] RBAC: Authorizing `identity:get_user(user_id=c7a8c94b3b3b46fe938600b73aa75ee2)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:10.469856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0443e-b28f-4160-a836-3a5d21eadd6b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:10.471507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 542/1085] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:10 2022] GET /identity/v3/users/c7a8c94b3b3b46fe938600b73aa75ee2 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:10.477109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0443e-b28f-4160-a836-3a5d21eadd6b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.479601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:10.541210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab7e203-46fc-4966-8f41-4d90756d9bfe None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.573127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.573812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.574088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.574356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.574893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0443e-b28f-4160-a836-3a5d21eadd6b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.575053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:10.575397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:10.576442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:10.576769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:10.584760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 544/1086] 10.222.0.69 () {56 vars in 1115 bytes} [Fri May 20 09:01:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:10.590414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.610450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26a6f8-b377-40aa-83c0-4a4ca2c5f33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.611352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a6f8-b377-40aa-83c0-4a4ca2c5f33d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.611635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a6f8-b377-40aa-83c0-4a4ca2c5f33d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.611908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26a6f8-b377-40aa-83c0-4a4ca2c5f33d None admin] PATH_INFO: `/v3/projects/ca30252be2274a08a1d81bca2de0ec13` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.613193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26a6f8-b377-40aa-83c0-4a4ca2c5f33d None admin] RBAC: Authorizing `identity:get_project(project_id=ca30252be2274a08a1d81bca2de0ec13)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:10.614656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26a6f8-b377-40aa-83c0-4a4ca2c5f33d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:10.616462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 543/1087] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:10 2022] GET /identity/v3/projects/ca30252be2274a08a1d81bca2de0ec13 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:10.622230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26a6f8-b377-40aa-83c0-4a4ca2c5f33d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.687140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b0a9d-b7fb-4607-8396-6a9d835baf34 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.688715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:10.717805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26a6f8-b377-40aa-83c0-4a4ca2c5f33d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.722447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.723241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.723549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.723824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.734544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:10.736019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:10.750092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dccfab-018f-4ea8-85a0-37aa3d8746b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.751011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dccfab-018f-4ea8-85a0-37aa3d8746b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.751319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dccfab-018f-4ea8-85a0-37aa3d8746b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.751598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dccfab-018f-4ea8-85a0-37aa3d8746b7 None admin] PATH_INFO: `/v3/projects/ca30252be2274a08a1d81bca2de0ec13/users/c7a8c94b3b3b46fe938600b73aa75ee2/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:01:10.752645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:10.753668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:10.754952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:01:10.754952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:01:10.754952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:10.754952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:10.754952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 545/1088] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:01:10.755295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dccfab-018f-4ea8-85a0-37aa3d8746b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca30252be2274a08a1d81bca2de0ec13, user_id=c7a8c94b3b3b46fe938600b73aa75ee2, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:10.756365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dccfab-018f-4ea8-85a0-37aa3d8746b7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:10.759539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.770785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 544/1089] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:01:10 2022] PUT /identity/v3/projects/ca30252be2274a08a1d81bca2de0ec13/users/c7a8c94b3b3b46fe938600b73aa75ee2/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:01:10.869272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721148f2-e6a2-4d46-bd07-cf1d481b9326 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:10.905914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:10.906589 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:10.906878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:10.907139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:10.907913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:10.908217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:10.909616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:10.923178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 546/1090] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:10.929179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dccfab-018f-4ea8-85a0-37aa3d8746b7 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:10.965137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:11.047778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dccfab-018f-4ea8-85a0-37aa3d8746b7 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:11.080165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:11.080952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:11.081205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:11.081496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:11.088863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:11.090300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:01:11.103884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:11.105352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:11.106518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 545/1091] 10.222.0.69 () {56 vars in 1093 bytes} [Fri May 20 09:01:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:11.111155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:11.129271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:11.216156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e74891e-08ac-456f-bae5-218b55ecc367 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:11.250617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:11.251294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:11.251580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:11.251850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:11.252684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:11.252987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:11.254404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:11.274329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 547/1092] 10.222.0.69 () {56 vars in 1133 bytes} [Fri May 20 09:01:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:11.280233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:11.372643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:11.383565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948dfa3-bc4b-48d6-b6a5-67d8bfd85457 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:11.420382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a042f7-d3ea-4d1b-84bf-bfddee3c6210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:11.421028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a042f7-d3ea-4d1b-84bf-bfddee3c6210 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:11.421311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a042f7-d3ea-4d1b-84bf-bfddee3c6210 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:11.421577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a042f7-d3ea-4d1b-84bf-bfddee3c6210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:11.492204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a042f7-d3ea-4d1b-84bf-bfddee3c6210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:11.493592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a042f7-d3ea-4d1b-84bf-bfddee3c6210 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:11.583390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:11.595458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 546/1093] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:11 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:11.601462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:11.674325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:11.702690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ac3a5-9347-49e6-aea1-130bc5f700df None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:11.735913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:11.736683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:11.736963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:11.737229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:11.748008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:11.749386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:01:11.766107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:11.767025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:11.772983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:01:11.772983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:01:11.772983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:01:11.772983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:11.772983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 548/1094] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:11.773400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a042f7-d3ea-4d1b-84bf-bfddee3c6210 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:11.872938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a042f7-d3ea-4d1b-84bf-bfddee3c6210 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:11.909063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:11.909755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:11.910012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:11.910282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:11.911005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:11.911318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:11.912740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:11.924403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 547/1095] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:11.930205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:12.029245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0998-76ae-4f71-9f52-0e374d84eccd None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:12.052140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:12.063574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:12.064362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:12.064646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:12.064926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:12.071703 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:12.073150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:01:12.087099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:12.088058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:12.088954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 549/1096] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:01:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:12.094281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:12.144648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:12.200434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc965db-ef32-41f8-8966-eef38ba17447 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:12.234317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48b012a-ac81-4f1a-9ed3-27a8517c0a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:12.234963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b012a-ac81-4f1a-9ed3-27a8517c0a5f None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:12.235244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b012a-ac81-4f1a-9ed3-27a8517c0a5f None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:12.235543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48b012a-ac81-4f1a-9ed3-27a8517c0a5f None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:12.236331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b012a-ac81-4f1a-9ed3-27a8517c0a5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:12.236610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b012a-ac81-4f1a-9ed3-27a8517c0a5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:12.238082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48b012a-ac81-4f1a-9ed3-27a8517c0a5f None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:12.250032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 548/1097] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:01:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:12.256246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:12.360130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcbd591-74b3-4ee8-ae48-93f9024c79e9 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:12.396843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e898d813-fd1a-410c-b9fb-e05fb9f98076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:12.397494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d813-fd1a-410c-b9fb-e05fb9f98076 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:12.397778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d813-fd1a-410c-b9fb-e05fb9f98076 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:12.398051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e898d813-fd1a-410c-b9fb-e05fb9f98076 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:12.398889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898d813-fd1a-410c-b9fb-e05fb9f98076 None admin] RBAC: Adding query filter params (role.id=8bdf6fdbab194849a9ac0170382b070b, scope.project.id=569782752acb4085ad742330af999bc2, user.id=04cd27bb637a404d827d384196eabfb1) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:12.399200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898d813-fd1a-410c-b9fb-e05fb9f98076 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:12.400693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e898d813-fd1a-410c-b9fb-e05fb9f98076 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:12.407031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 550/1098] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:12 2022] GET /identity/v3/role_assignments?user.id=04cd27bb637a404d827d384196eabfb1&scope.project.id=569782752acb4085ad742330af999bc2&role.id=8bdf6fdbab194849a9ac0170382b070b => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:12.959984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad4127-40b4-4326-82fd-b7169bb1bee1 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:12.960532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad4127-40b4-4326-82fd-b7169bb1bee1 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:12.960532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ad4127-40b4-4326-82fd-b7169bb1bee1 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:12.961944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 549/1099] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:12.967436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c9cf-8312-4d89-8e04-90d419d66332 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:12.967770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c9cf-8312-4d89-8e04-90d419d66332 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:12.968053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82c9cf-8312-4d89-8e04-90d419d66332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:12.986416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1e82c9cf-8312-4d89-8e04-90d419d66332 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:12.987645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1e82c9cf-8312-4d89-8e04-90d419d66332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:13.036298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:13.060309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 551/1100] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:13.083374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad4127-40b4-4326-82fd-b7169bb1bee1 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:13.188387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ad4127-40b4-4326-82fd-b7169bb1bee1 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:13.201677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:13.232250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6e21c-1fd2-4fc5-b0a2-70a1dbe54171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:13.233050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6e21c-1fd2-4fc5-b0a2-70a1dbe54171 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:13.234502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6e21c-1fd2-4fc5-b0a2-70a1dbe54171 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:13.234786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6e21c-1fd2-4fc5-b0a2-70a1dbe54171 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:13.247559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa6e21c-1fd2-4fc5-b0a2-70a1dbe54171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-3fa6e21c-1fd2-4fc5-b0a2-70a1dbe54171 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:13.254947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:13.256034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:13.257139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 550/1101] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:01:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:13.262347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82c9cf-8312-4d89-8e04-90d419d66332 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:13.371196 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82c9cf-8312-4d89-8e04-90d419d66332 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:13.407431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:13.408091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:13.408373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:13.408608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:13.409388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:13.409693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:13.411080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:13.419761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 552/1102] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:01:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:13.426475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6e21c-1fd2-4fc5-b0a2-70a1dbe54171 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:13.529008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6e21c-1fd2-4fc5-b0a2-70a1dbe54171 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:13.564739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f3d07-1c09-479b-878d-e47f3fbc26cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:13.565606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f3d07-1c09-479b-878d-e47f3fbc26cc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:13.565892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f3d07-1c09-479b-878d-e47f3fbc26cc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:13.566163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f3d07-1c09-479b-878d-e47f3fbc26cc None admin] PATH_INFO: `/v3/users/c7a8c94b3b3b46fe938600b73aa75ee2` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:13.567499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f3d07-1c09-479b-878d-e47f3fbc26cc None admin] RBAC: Authorizing `identity:get_user(user_id=c7a8c94b3b3b46fe938600b73aa75ee2)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:13.569063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f3d07-1c09-479b-878d-e47f3fbc26cc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:13.570680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 551/1103] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:13 2022] GET /identity/v3/users/c7a8c94b3b3b46fe938600b73aa75ee2 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:13.576420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:13.630163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:13.698925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ffec0e-2dd0-4a06-bb6e-cb62f815c13b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:13.739800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5261f-100b-46f2-a6ab-7da16d04e5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:13.740684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5261f-100b-46f2-a6ab-7da16d04e5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:13.741065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5261f-100b-46f2-a6ab-7da16d04e5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:13.741417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b5261f-100b-46f2-a6ab-7da16d04e5e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:13.816089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5261f-100b-46f2-a6ab-7da16d04e5e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:13.817535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b5261f-100b-46f2-a6ab-7da16d04e5e8 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:13.864729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:13.924918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 553/1104] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:13.931165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f3d07-1c09-479b-878d-e47f3fbc26cc None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:13.975108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:14.040116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f3d07-1c09-479b-878d-e47f3fbc26cc None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:14.076063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08be29dd-55cb-40cf-9a67-c9c984f1c6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:14.076915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be29dd-55cb-40cf-9a67-c9c984f1c6ee None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:14.077188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be29dd-55cb-40cf-9a67-c9c984f1c6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:14.077468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08be29dd-55cb-40cf-9a67-c9c984f1c6ee None admin] PATH_INFO: `/v3/projects/ca30252be2274a08a1d81bca2de0ec13` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:14.078742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be29dd-55cb-40cf-9a67-c9c984f1c6ee None admin] RBAC: Authorizing `identity:get_project(project_id=ca30252be2274a08a1d81bca2de0ec13)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:14.080206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08be29dd-55cb-40cf-9a67-c9c984f1c6ee None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:14.081892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 552/1105] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:13 2022] GET /identity/v3/projects/ca30252be2274a08a1d81bca2de0ec13 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:14.088321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5261f-100b-46f2-a6ab-7da16d04e5e8 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:14.197387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b5261f-100b-46f2-a6ab-7da16d04e5e8 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:14.231987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9940fedc-57b1-49c7-9cb0-1495160dfa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:14.232614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940fedc-57b1-49c7-9cb0-1495160dfa27 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:14.232896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940fedc-57b1-49c7-9cb0-1495160dfa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:14.233167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9940fedc-57b1-49c7-9cb0-1495160dfa27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:14.233988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940fedc-57b1-49c7-9cb0-1495160dfa27 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=ca30252be2274a08a1d81bca2de0ec13, user.id=c7a8c94b3b3b46fe938600b73aa75ee2) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:14.234294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940fedc-57b1-49c7-9cb0-1495160dfa27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:14.235766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9940fedc-57b1-49c7-9cb0-1495160dfa27 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:14.250322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 554/1106] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:14 2022] GET /identity/v3/role_assignments?user.id=c7a8c94b3b3b46fe938600b73aa75ee2&scope.project.id=ca30252be2274a08a1d81bca2de0ec13&role.id=6dca675ade814980a252f800fb4fa3ad => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:16.434212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0c77-3598-4573-b690-a320a1e312dc None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:16.435111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0c77-3598-4573-b690-a320a1e312dc None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:16.435111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3b0c77-3598-4573-b690-a320a1e312dc None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:16.437547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 553/1107] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:16.442989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098464-9c37-4d61-8886-9f88be81bce0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:16.443494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098464-9c37-4d61-8886-9f88be81bce0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:16.443593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098464-9c37-4d61-8886-9f88be81bce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:16.450482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:16.465697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e4098464-9c37-4d61-8886-9f88be81bce0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:16.466882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e4098464-9c37-4d61-8886-9f88be81bce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:16.536401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 555/1108] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:16.561174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b0c77-3598-4573-b690-a320a1e312dc None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:16.684514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3b0c77-3598-4573-b690-a320a1e312dc None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:16.691902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:16.723589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a155e-3be6-4945-bc7f-70dd21556208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:16.724392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a155e-3be6-4945-bc7f-70dd21556208 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:16.724709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a155e-3be6-4945-bc7f-70dd21556208 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:16.724988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a155e-3be6-4945-bc7f-70dd21556208 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:16.726287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a155e-3be6-4945-bc7f-70dd21556208 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:16.727782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a155e-3be6-4945-bc7f-70dd21556208 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:16.729402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 554/1109] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:16.736011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4098464-9c37-4d61-8886-9f88be81bce0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:16.845702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4098464-9c37-4d61-8886-9f88be81bce0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:16.880465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee945034-c043-428e-b57d-441a58736376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:16.881164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945034-c043-428e-b57d-441a58736376 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:16.881443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945034-c043-428e-b57d-441a58736376 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:16.881716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee945034-c043-428e-b57d-441a58736376 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:16.882428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee945034-c043-428e-b57d-441a58736376 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:16.883911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee945034-c043-428e-b57d-441a58736376 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:16.890666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:16.906481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 556/1110] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 09:01:16 2022] POST /identity/v3/users => generated 286 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:19.153850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e481b8-7a46-4add-a9e4-b131e7491e1f None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:19.154390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e481b8-7a46-4add-a9e4-b131e7491e1f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:19.154390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e481b8-7a46-4add-a9e4-b131e7491e1f None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:19.155723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 555/1111] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:19.162176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a93c1-ff5d-4621-8f48-c8a6e3980996 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:19.162895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a93c1-ff5d-4621-8f48-c8a6e3980996 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:19.163040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a93c1-ff5d-4621-8f48-c8a6e3980996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:19.187389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-ed4a93c1-ff5d-4621-8f48-c8a6e3980996 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:19.188582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ed4a93c1-ff5d-4621-8f48-c8a6e3980996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:19.264882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 557/1112] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:19.288310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e481b8-7a46-4add-a9e4-b131e7491e1f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:19.393549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e481b8-7a46-4add-a9e4-b131e7491e1f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:19.428573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7aab-dd9c-4e99-a94b-7ca0ff831c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:19.429366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7aab-dd9c-4e99-a94b-7ca0ff831c54 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:19.429648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7aab-dd9c-4e99-a94b-7ca0ff831c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:19.429921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffb7aab-dd9c-4e99-a94b-7ca0ff831c54 None admin] PATH_INFO: `/v3/roles/2e550a6a96bb4804926c6213a5af7585` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:19.431462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb7aab-dd9c-4e99-a94b-7ca0ff831c54 None admin] RBAC: Authorizing `identity:get_role(role_id=2e550a6a96bb4804926c6213a5af7585)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:19.432849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffb7aab-dd9c-4e99-a94b-7ca0ff831c54 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:19.434237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 556/1113] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:19 2022] GET /identity/v3/roles/2e550a6a96bb4804926c6213a5af7585 => generated 223 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:19.440899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a93c1-ff5d-4621-8f48-c8a6e3980996 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:19.556831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a93c1-ff5d-4621-8f48-c8a6e3980996 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:19.576781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:19.601064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215b47a-9f3c-438b-95a6-0385d3dab8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:19.601937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b47a-9f3c-438b-95a6-0385d3dab8ec None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:19.602210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b47a-9f3c-438b-95a6-0385d3dab8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:19.602479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215b47a-9f3c-438b-95a6-0385d3dab8ec None admin] PATH_INFO: `/v3/users/3e880fa6249e4e0a8665173e6006967f` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:19.615355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215b47a-9f3c-438b-95a6-0385d3dab8ec None admin] RBAC: Authorizing `identity:get_user(user_id=3e880fa6249e4e0a8665173e6006967f)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:19.616772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215b47a-9f3c-438b-95a6-0385d3dab8ec None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:19.618423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 558/1114] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:19 2022] GET /identity/v3/users/3e880fa6249e4e0a8665173e6006967f => generated 286 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:19.624351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7aab-dd9c-4e99-a94b-7ca0ff831c54 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:19.674480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:19.739032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffb7aab-dd9c-4e99-a94b-7ca0ff831c54 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:19.776044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573cb8c-8796-4544-b0d5-25b6e6d34e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:19.776991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573cb8c-8796-4544-b0d5-25b6e6d34e8c None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:19.777341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573cb8c-8796-4544-b0d5-25b6e6d34e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:19.777677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573cb8c-8796-4544-b0d5-25b6e6d34e8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:19.846719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573cb8c-8796-4544-b0d5-25b6e6d34e8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:19.848215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7573cb8c-8796-4544-b0d5-25b6e6d34e8c None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:19.896995 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:19.956821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 557/1115] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:19.963744 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215b47a-9f3c-438b-95a6-0385d3dab8ec None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:20.083610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215b47a-9f3c-438b-95a6-0385d3dab8ec None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:20.119238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2852493-6b1c-4d30-80af-3557d40b2e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:20.120110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2852493-6b1c-4d30-80af-3557d40b2e6f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:20.120386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2852493-6b1c-4d30-80af-3557d40b2e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:20.120663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2852493-6b1c-4d30-80af-3557d40b2e6f None admin] PATH_INFO: `/v3/projects/ca30252be2274a08a1d81bca2de0ec13` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:20.121898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2852493-6b1c-4d30-80af-3557d40b2e6f None admin] RBAC: Authorizing `identity:get_project(project_id=ca30252be2274a08a1d81bca2de0ec13)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:20.123389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2852493-6b1c-4d30-80af-3557d40b2e6f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:20.125056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 559/1116] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:19 2022] GET /identity/v3/projects/ca30252be2274a08a1d81bca2de0ec13 => generated 311 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:20.134794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573cb8c-8796-4544-b0d5-25b6e6d34e8c None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:20.242514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573cb8c-8796-4544-b0d5-25b6e6d34e8c None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:20.277446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741850f8-a3c8-48b2-ade6-753d8b2abe4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:20.278084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741850f8-a3c8-48b2-ade6-753d8b2abe4b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:20.278361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741850f8-a3c8-48b2-ade6-753d8b2abe4b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:20.278635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741850f8-a3c8-48b2-ade6-753d8b2abe4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:20.279488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741850f8-a3c8-48b2-ade6-753d8b2abe4b None admin] RBAC: Adding query filter params (role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=ca30252be2274a08a1d81bca2de0ec13, user.id=3e880fa6249e4e0a8665173e6006967f) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:20.279798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741850f8-a3c8-48b2-ade6-753d8b2abe4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:20.281226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741850f8-a3c8-48b2-ade6-753d8b2abe4b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:20.286695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 558/1117] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:20 2022] GET /identity/v3/role_assignments?user.id=3e880fa6249e4e0a8665173e6006967f&scope.project.id=ca30252be2274a08a1d81bca2de0ec13&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:22.457829 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a27db-1ef7-49d4-8e07-b07c0f7cafc5 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:22.458837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a27db-1ef7-49d4-8e07-b07c0f7cafc5 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:22.459003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661a27db-1ef7-49d4-8e07-b07c0f7cafc5 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:22.461215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 560/1118] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:22.470682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7df7b-1902-445d-942f-fce7a1537145 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:22.471460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7df7b-1902-445d-942f-fce7a1537145 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:22.472114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef7df7b-1902-445d-942f-fce7a1537145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:22.491355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:22.496871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6ef7df7b-1902-445d-942f-fce7a1537145 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:22.498062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6ef7df7b-1902-445d-942f-fce7a1537145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:22.566367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 559/1119] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:22.590349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a27db-1ef7-49d4-8e07-b07c0f7cafc5 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:22.646477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:22.703000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661a27db-1ef7-49d4-8e07-b07c0f7cafc5 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:22.737322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa36b7-fc9f-46f8-b04e-3f51f0b804d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:22.738100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa36b7-fc9f-46f8-b04e-3f51f0b804d1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:22.738376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa36b7-fc9f-46f8-b04e-3f51f0b804d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:22.738644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa36b7-fc9f-46f8-b04e-3f51f0b804d1 None admin] PATH_INFO: `/v3/roles/2e550a6a96bb4804926c6213a5af7585` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:22.740157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa36b7-fc9f-46f8-b04e-3f51f0b804d1 None admin] RBAC: Authorizing `identity:get_role(role_id=2e550a6a96bb4804926c6213a5af7585)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:22.741533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa36b7-fc9f-46f8-b04e-3f51f0b804d1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:22.742905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 561/1120] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:22 2022] GET /identity/v3/roles/2e550a6a96bb4804926c6213a5af7585 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:22.752116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7df7b-1902-445d-942f-fce7a1537145 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:22.865578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef7df7b-1902-445d-942f-fce7a1537145 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:22.902375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04051ede-ee77-4d2b-8115-b7ffed0b1d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:22.903247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04051ede-ee77-4d2b-8115-b7ffed0b1d3a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:22.903560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04051ede-ee77-4d2b-8115-b7ffed0b1d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:22.903829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04051ede-ee77-4d2b-8115-b7ffed0b1d3a None admin] PATH_INFO: `/v3/users/3e880fa6249e4e0a8665173e6006967f` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:22.905244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04051ede-ee77-4d2b-8115-b7ffed0b1d3a None admin] RBAC: Authorizing `identity:get_user(user_id=3e880fa6249e4e0a8665173e6006967f)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:22.906701 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04051ede-ee77-4d2b-8115-b7ffed0b1d3a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:22.908358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 560/1121] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:22 2022] GET /identity/v3/users/3e880fa6249e4e0a8665173e6006967f => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:22.914766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa36b7-fc9f-46f8-b04e-3f51f0b804d1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:23.014963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa36b7-fc9f-46f8-b04e-3f51f0b804d1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:23.028974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:23.049548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ca891f-7ec3-42ce-8f4a-173b97fe0201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:23.050388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca891f-7ec3-42ce-8f4a-173b97fe0201 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:23.050666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca891f-7ec3-42ce-8f4a-173b97fe0201 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:23.050935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ca891f-7ec3-42ce-8f4a-173b97fe0201 None admin] PATH_INFO: `/v3/projects/ca30252be2274a08a1d81bca2de0ec13` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:23.052170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca891f-7ec3-42ce-8f4a-173b97fe0201 None admin] RBAC: Authorizing `identity:get_project(project_id=ca30252be2274a08a1d81bca2de0ec13)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:23.053636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ca891f-7ec3-42ce-8f4a-173b97fe0201 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:23.055291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 562/1122] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:22 2022] GET /identity/v3/projects/ca30252be2274a08a1d81bca2de0ec13 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:01:23.064397 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04051ede-ee77-4d2b-8115-b7ffed0b1d3a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:23.070470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:23.175660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04051ede-ee77-4d2b-8115-b7ffed0b1d3a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:23.210074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a309d15-45d3-4db5-961c-85bb57be1856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:23.210954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309d15-45d3-4db5-961c-85bb57be1856 None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:23.211228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309d15-45d3-4db5-961c-85bb57be1856 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:23.211534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a309d15-45d3-4db5-961c-85bb57be1856 None admin] PATH_INFO: `/v3/projects/ca30252be2274a08a1d81bca2de0ec13/users/3e880fa6249e4e0a8665173e6006967f/roles/2e550a6a96bb4804926c6213a5af7585` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:23.213877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a309d15-45d3-4db5-961c-85bb57be1856 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca30252be2274a08a1d81bca2de0ec13, user_id=3e880fa6249e4e0a8665173e6006967f, role_id=2e550a6a96bb4804926c6213a5af7585)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:23.215349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a309d15-45d3-4db5-961c-85bb57be1856 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:23.234011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 561/1123] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:01:23 2022] PUT /identity/v3/projects/ca30252be2274a08a1d81bca2de0ec13/users/3e880fa6249e4e0a8665173e6006967f/roles/2e550a6a96bb4804926c6213a5af7585 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:01:25.415305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f6c15-3840-497d-bf47-bf68c395589d None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:25.415848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f6c15-3840-497d-bf47-bf68c395589d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:25.415848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6f6c15-3840-497d-bf47-bf68c395589d None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:25.418000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 563/1124] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:25.424818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87bcb2-c8cc-48b1-bf4a-8ccd1a0298d9 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:25.425512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87bcb2-c8cc-48b1-bf4a-8ccd1a0298d9 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:25.425695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f87bcb2-c8cc-48b1-bf4a-8ccd1a0298d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:25.444881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6f87bcb2-c8cc-48b1-bf4a-8ccd1a0298d9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:25.446069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6f87bcb2-c8cc-48b1-bf4a-8ccd1a0298d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:25.510644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:25.521310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 562/1125] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:25.545827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f6c15-3840-497d-bf47-bf68c395589d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:25.659565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6f6c15-3840-497d-bf47-bf68c395589d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:25.694925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006af7f-b488-45e4-9ace-c6429cfeba60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:25.695723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006af7f-b488-45e4-9ace-c6429cfeba60 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:25.695978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006af7f-b488-45e4-9ace-c6429cfeba60 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:25.696251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006af7f-b488-45e4-9ace-c6429cfeba60 None admin] PATH_INFO: `/v3/roles/2e550a6a96bb4804926c6213a5af7585` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:25.697709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006af7f-b488-45e4-9ace-c6429cfeba60 None admin] RBAC: Authorizing `identity:get_role(role_id=2e550a6a96bb4804926c6213a5af7585)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:25.699114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006af7f-b488-45e4-9ace-c6429cfeba60 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:25.700498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 564/1126] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:25 2022] GET /identity/v3/roles/2e550a6a96bb4804926c6213a5af7585 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:25.709690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87bcb2-c8cc-48b1-bf4a-8ccd1a0298d9 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:25.816439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f87bcb2-c8cc-48b1-bf4a-8ccd1a0298d9 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:25.853235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec0576-b1e4-464f-81f4-6b8f3c1c2c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:25.854106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec0576-b1e4-464f-81f4-6b8f3c1c2c55 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:25.854382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec0576-b1e4-464f-81f4-6b8f3c1c2c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:25.854652 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec0576-b1e4-464f-81f4-6b8f3c1c2c55 None admin] PATH_INFO: `/v3/users/3e880fa6249e4e0a8665173e6006967f` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:25.855983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec0576-b1e4-464f-81f4-6b8f3c1c2c55 None admin] RBAC: Authorizing `identity:get_user(user_id=3e880fa6249e4e0a8665173e6006967f)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:25.857461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec0576-b1e4-464f-81f4-6b8f3c1c2c55 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:25.859099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 563/1127] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:25 2022] GET /identity/v3/users/3e880fa6249e4e0a8665173e6006967f => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:25.865279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006af7f-b488-45e4-9ace-c6429cfeba60 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:25.919281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:25.971887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006af7f-b488-45e4-9ace-c6429cfeba60 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:26.006301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706579e8-9832-4fd0-bfa6-497978cc9879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:26.006931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706579e8-9832-4fd0-bfa6-497978cc9879 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:26.007213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706579e8-9832-4fd0-bfa6-497978cc9879 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:26.007520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706579e8-9832-4fd0-bfa6-497978cc9879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:26.075301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706579e8-9832-4fd0-bfa6-497978cc9879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:26.076767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706579e8-9832-4fd0-bfa6-497978cc9879 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:26.123747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:26.177760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 565/1128] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:26.184917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec0576-b1e4-464f-81f4-6b8f3c1c2c55 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:26.255846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:26.295166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec0576-b1e4-464f-81f4-6b8f3c1c2c55 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:26.329449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e23753-226b-446f-b8e2-aacd05326f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:26.330295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e23753-226b-446f-b8e2-aacd05326f61 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:26.330572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e23753-226b-446f-b8e2-aacd05326f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:26.330839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e23753-226b-446f-b8e2-aacd05326f61 None admin] PATH_INFO: `/v3/projects/ca30252be2274a08a1d81bca2de0ec13` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:26.332062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e23753-226b-446f-b8e2-aacd05326f61 None admin] RBAC: Authorizing `identity:get_project(project_id=ca30252be2274a08a1d81bca2de0ec13)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:26.333502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e23753-226b-446f-b8e2-aacd05326f61 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:26.335202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 564/1129] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:26 2022] GET /identity/v3/projects/ca30252be2274a08a1d81bca2de0ec13 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:26.344611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706579e8-9832-4fd0-bfa6-497978cc9879 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:26.446042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706579e8-9832-4fd0-bfa6-497978cc9879 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:26.480315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd59c52-f228-4451-841e-401146654548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:26.480942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59c52-f228-4451-841e-401146654548 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:26.481213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59c52-f228-4451-841e-401146654548 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:26.481483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd59c52-f228-4451-841e-401146654548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:26.482315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd59c52-f228-4451-841e-401146654548 None admin] RBAC: Adding query filter params (role.id=2e550a6a96bb4804926c6213a5af7585, scope.project.id=ca30252be2274a08a1d81bca2de0ec13, user.id=3e880fa6249e4e0a8665173e6006967f) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:26.482623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd59c52-f228-4451-841e-401146654548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:26.484104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd59c52-f228-4451-841e-401146654548 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:26.489981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 566/1130] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:26 2022] GET /identity/v3/role_assignments?user.id=3e880fa6249e4e0a8665173e6006967f&scope.project.id=ca30252be2274a08a1d81bca2de0ec13&role.id=2e550a6a96bb4804926c6213a5af7585 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:28.672757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025b541-8bc9-42ba-b30c-e7659dfb834c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:28.673611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025b541-8bc9-42ba-b30c-e7659dfb834c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:28.673611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025b541-8bc9-42ba-b30c-e7659dfb834c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:28.675912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 565/1131] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:28.681304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fe70d-14fc-4f20-ba4d-528109dab143 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:28.681727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fe70d-14fc-4f20-ba4d-528109dab143 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:28.681818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fe70d-14fc-4f20-ba4d-528109dab143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:28.700322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-192fe70d-14fc-4f20-ba4d-528109dab143 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:28.701485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-192fe70d-14fc-4f20-ba4d-528109dab143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:28.711388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:28.788589 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 567/1132] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:28.823712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025b541-8bc9-42ba-b30c-e7659dfb834c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:28.934361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025b541-8bc9-42ba-b30c-e7659dfb834c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:28.963144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:28.970947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491fbcbb-8096-48b9-a8bc-4851d5c02d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:28.971780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fbcbb-8096-48b9-a8bc-4851d5c02d49 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:28.972062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fbcbb-8096-48b9-a8bc-4851d5c02d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:28.972337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491fbcbb-8096-48b9-a8bc-4851d5c02d49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:28.973608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fbcbb-8096-48b9-a8bc-4851d5c02d49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:28.975102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491fbcbb-8096-48b9-a8bc-4851d5c02d49 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:28.976776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 566/1133] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:28.983226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fe70d-14fc-4f20-ba4d-528109dab143 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:29.087169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fe70d-14fc-4f20-ba4d-528109dab143 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:29.127624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d756459-601d-4892-8200-381ce7563ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:29.128273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756459-601d-4892-8200-381ce7563ee0 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:29.128556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756459-601d-4892-8200-381ce7563ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:29.128827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d756459-601d-4892-8200-381ce7563ee0 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:29.129529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d756459-601d-4892-8200-381ce7563ee0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:29.130995 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d756459-601d-4892-8200-381ce7563ee0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:29.154937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:29.172739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 568/1134] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:01:28 2022] POST /identity/v3/projects => generated 311 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:31.378623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435401-e405-4eae-93ec-bab6c6c412c3 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:31.379654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435401-e405-4eae-93ec-bab6c6c412c3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:31.379654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435401-e405-4eae-93ec-bab6c6c412c3 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:31.381777 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 567/1135] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:31.387421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48040bed-9b3a-4567-86f7-1b45ca7623ff None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:31.387860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48040bed-9b3a-4567-86f7-1b45ca7623ff None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:31.387952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48040bed-9b3a-4567-86f7-1b45ca7623ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:31.406559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-48040bed-9b3a-4567-86f7-1b45ca7623ff None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:31.407750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-48040bed-9b3a-4567-86f7-1b45ca7623ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:31.493206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 569/1136] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:31.518109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435401-e405-4eae-93ec-bab6c6c412c3 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:31.629123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435401-e405-4eae-93ec-bab6c6c412c3 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:31.664081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597be776-02e4-4257-a9ea-a3f582a364c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:31.664862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be776-02e4-4257-a9ea-a3f582a364c9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:31.665149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be776-02e4-4257-a9ea-a3f582a364c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:31.665417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597be776-02e4-4257-a9ea-a3f582a364c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:31.666646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597be776-02e4-4257-a9ea-a3f582a364c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:31.668145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597be776-02e4-4257-a9ea-a3f582a364c9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:31.669825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 568/1137] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:01:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:31.676569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48040bed-9b3a-4567-86f7-1b45ca7623ff None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:31.797949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48040bed-9b3a-4567-86f7-1b45ca7623ff None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:31.823830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:31.834056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a2d17-4915-4e7d-a4cc-de62eef03aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:31.834756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a2d17-4915-4e7d-a4cc-de62eef03aaf None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:31.835027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a2d17-4915-4e7d-a4cc-de62eef03aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:31.835292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a2d17-4915-4e7d-a4cc-de62eef03aaf None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:31.836027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a2d17-4915-4e7d-a4cc-de62eef03aaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:31.837450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a2d17-4915-4e7d-a4cc-de62eef03aaf None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:31.864023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 570/1138] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 09:01:31 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:34.104473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d560bf-16a9-45e5-b0d7-f955b0dd0709 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:34.105001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d560bf-16a9-45e5-b0d7-f955b0dd0709 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:34.105001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d560bf-16a9-45e5-b0d7-f955b0dd0709 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:34.106802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 569/1139] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:34.113357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bd65-f820-428d-9619-87199b4de402 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:34.114084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bd65-f820-428d-9619-87199b4de402 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:34.114234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d8bd65-f820-428d-9619-87199b4de402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:34.139164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-42d8bd65-f820-428d-9619-87199b4de402 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:34.140730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-42d8bd65-f820-428d-9619-87199b4de402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:34.225335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 571/1140] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:34.251471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d560bf-16a9-45e5-b0d7-f955b0dd0709 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:34.331570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:34.372216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d560bf-16a9-45e5-b0d7-f955b0dd0709 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:34.411133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232fbfa-9dc7-477c-8c57-8ed487c37d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:34.411973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232fbfa-9dc7-477c-8c57-8ed487c37d76 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:34.412257 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232fbfa-9dc7-477c-8c57-8ed487c37d76 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:34.412525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232fbfa-9dc7-477c-8c57-8ed487c37d76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:34.423637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232fbfa-9dc7-477c-8c57-8ed487c37d76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-1232fbfa-9dc7-477c-8c57-8ed487c37d76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:34.435217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:34.436440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:34.437300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 570/1141] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:34.443533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8bd65-f820-428d-9619-87199b4de402 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:34.573699 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d8bd65-f820-428d-9619-87199b4de402 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:34.586615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:34.610749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:34.611432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:34.611708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:34.611975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:34.612720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:34.613024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:34.614379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:34.622720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 572/1142] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:01:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:34.628594 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232fbfa-9dc7-477c-8c57-8ed487c37d76 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:34.754137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232fbfa-9dc7-477c-8c57-8ed487c37d76 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:34.783233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:34.794047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc669de-fbc1-4726-b1a1-8287ddb03fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:34.794922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc669de-fbc1-4726-b1a1-8287ddb03fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:34.795203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc669de-fbc1-4726-b1a1-8287ddb03fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:34.795501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc669de-fbc1-4726-b1a1-8287ddb03fc6 None admin] PATH_INFO: `/v3/users/92255253749d405e938df5c35d1a54df` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:34.808976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc669de-fbc1-4726-b1a1-8287ddb03fc6 None admin] RBAC: Authorizing `identity:get_user(user_id=92255253749d405e938df5c35d1a54df)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:34.810461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc669de-fbc1-4726-b1a1-8287ddb03fc6 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:34.812152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 571/1143] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:34 2022] GET /identity/v3/users/92255253749d405e938df5c35d1a54df => generated 286 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:34.818069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:34.919833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354abf44-3432-4bfc-8c91-762aababaad4 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:34.954488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1051eee-2883-47b4-a6a2-3c60f5ffd3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:34.955116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051eee-2883-47b4-a6a2-3c60f5ffd3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:34.955429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051eee-2883-47b4-a6a2-3c60f5ffd3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:34.955708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1051eee-2883-47b4-a6a2-3c60f5ffd3d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:34.998246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:35.032142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1051eee-2883-47b4-a6a2-3c60f5ffd3d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:35.033618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1051eee-2883-47b4-a6a2-3c60f5ffd3d9 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:35.133164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 573/1144] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:35.139449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc669de-fbc1-4726-b1a1-8287ddb03fc6 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:35.248336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc669de-fbc1-4726-b1a1-8287ddb03fc6 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:35.282849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ff7b6-694f-4535-a393-3ef6db1b0c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:35.283727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ff7b6-694f-4535-a393-3ef6db1b0c91 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:35.284002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ff7b6-694f-4535-a393-3ef6db1b0c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:35.284274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ff7b6-694f-4535-a393-3ef6db1b0c91 None admin] PATH_INFO: `/v3/projects/25991028b0af4028a346dd89eaf22702` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:35.285483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ff7b6-694f-4535-a393-3ef6db1b0c91 None admin] RBAC: Authorizing `identity:get_project(project_id=25991028b0af4028a346dd89eaf22702)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:35.286920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ff7b6-694f-4535-a393-3ef6db1b0c91 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:35.289055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 572/1145] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:35 2022] GET /identity/v3/projects/25991028b0af4028a346dd89eaf22702 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:35.298920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1051eee-2883-47b4-a6a2-3c60f5ffd3d9 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:35.388125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:35.416030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1051eee-2883-47b4-a6a2-3c60f5ffd3d9 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:35.451017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef43f4d-55fe-460b-9c9c-ca242f344d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:35.452030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43f4d-55fe-460b-9c9c-ca242f344d08 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:35.452441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43f4d-55fe-460b-9c9c-ca242f344d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:35.452841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef43f4d-55fe-460b-9c9c-ca242f344d08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:35.454146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef43f4d-55fe-460b-9c9c-ca242f344d08 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=25991028b0af4028a346dd89eaf22702, user.id=92255253749d405e938df5c35d1a54df) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:35.454617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef43f4d-55fe-460b-9c9c-ca242f344d08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:35.456868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef43f4d-55fe-460b-9c9c-ca242f344d08 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:35.466188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 574/1146] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:35 2022] GET /identity/v3/role_assignments?user.id=92255253749d405e938df5c35d1a54df&scope.project.id=25991028b0af4028a346dd89eaf22702&role.id=6dca675ade814980a252f800fb4fa3ad => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:37.609479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd85c-a9a8-4c63-bc55-106c5a0b5a3d None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:37.611569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd85c-a9a8-4c63-bc55-106c5a0b5a3d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:37.611948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bd85c-a9a8-4c63-bc55-106c5a0b5a3d None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:37.617373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 573/1147] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:37.620263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5d152-8509-48d5-9522-6eab47212752 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:37.623434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5d152-8509-48d5-9522-6eab47212752 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:37.623434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5d152-8509-48d5-9522-6eab47212752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:37.642259 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b3a5d152-8509-48d5-9522-6eab47212752 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:37.643509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b3a5d152-8509-48d5-9522-6eab47212752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:37.714031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 575/1148] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:37.741653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bd85c-a9a8-4c63-bc55-106c5a0b5a3d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:37.809824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:37.858370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bd85c-a9a8-4c63-bc55-106c5a0b5a3d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:37.892304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636333a-dda6-4654-9bd9-0159ce7c625d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:37.893031 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636333a-dda6-4654-9bd9-0159ce7c625d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:37.893310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636333a-dda6-4654-9bd9-0159ce7c625d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:37.893573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636333a-dda6-4654-9bd9-0159ce7c625d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:37.904150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636333a-dda6-4654-9bd9-0159ce7c625d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-a636333a-dda6-4654-9bd9-0159ce7c625d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:37.911598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:37.912685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:37.913931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 574/1149] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:01:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:37.920256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5d152-8509-48d5-9522-6eab47212752 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:37.996133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:38.033280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5d152-8509-48d5-9522-6eab47212752 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:38.069303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:38.069990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:38.070272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:38.070551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:38.071305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:38.071637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:38.073014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:38.081139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 576/1150] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:01:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:38.087519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636333a-dda6-4654-9bd9-0159ce7c625d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:38.192810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636333a-dda6-4654-9bd9-0159ce7c625d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:38.205254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:38.228273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c86cee-dcbc-454e-85c2-3a5bd522bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:38.229132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c86cee-dcbc-454e-85c2-3a5bd522bee8 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:38.229407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c86cee-dcbc-454e-85c2-3a5bd522bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:38.229678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c86cee-dcbc-454e-85c2-3a5bd522bee8 None admin] PATH_INFO: `/v3/users/92255253749d405e938df5c35d1a54df` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:38.230976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c86cee-dcbc-454e-85c2-3a5bd522bee8 None admin] RBAC: Authorizing `identity:get_user(user_id=92255253749d405e938df5c35d1a54df)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:38.232449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c86cee-dcbc-454e-85c2-3a5bd522bee8 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:38.234105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 575/1151] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:38 2022] GET /identity/v3/users/92255253749d405e938df5c35d1a54df => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:38.243318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:38.353992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3938a83-eb92-4f4c-b961-1e893afd7d58 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:38.383775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:38.391018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f53c9-42e4-4ace-a5b7-fa8f1f7fe328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:38.391909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f53c9-42e4-4ace-a5b7-fa8f1f7fe328 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:38.392191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f53c9-42e4-4ace-a5b7-fa8f1f7fe328 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:38.392462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f53c9-42e4-4ace-a5b7-fa8f1f7fe328 None admin] PATH_INFO: `/v3/projects/25991028b0af4028a346dd89eaf22702` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:38.393655 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f53c9-42e4-4ace-a5b7-fa8f1f7fe328 None admin] RBAC: Authorizing `identity:get_project(project_id=25991028b0af4028a346dd89eaf22702)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:38.395092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9f53c9-42e4-4ace-a5b7-fa8f1f7fe328 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:38.396819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 577/1152] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:38 2022] GET /identity/v3/projects/25991028b0af4028a346dd89eaf22702 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:38.403088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c86cee-dcbc-454e-85c2-3a5bd522bee8 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:38.506129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c86cee-dcbc-454e-85c2-3a5bd522bee8 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:38.540727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74063889-e421-4c1c-9948-f69e08a2304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:38.541599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74063889-e421-4c1c-9948-f69e08a2304a None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:38.541874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74063889-e421-4c1c-9948-f69e08a2304a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:38.542141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74063889-e421-4c1c-9948-f69e08a2304a None admin] PATH_INFO: `/v3/projects/25991028b0af4028a346dd89eaf22702/users/92255253749d405e938df5c35d1a54df/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:38.544357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74063889-e421-4c1c-9948-f69e08a2304a None admin] RBAC: Authorizing `identity:create_grant(project_id=25991028b0af4028a346dd89eaf22702, user_id=92255253749d405e938df5c35d1a54df, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:38.545781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74063889-e421-4c1c-9948-f69e08a2304a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:38.566323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 576/1153] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:01:38 2022] PUT /identity/v3/projects/25991028b0af4028a346dd89eaf22702/users/92255253749d405e938df5c35d1a54df/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:01:40.701716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ffdac-484a-40ac-a187-0842d546e8b7 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:40.702311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ffdac-484a-40ac-a187-0842d546e8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:40.702311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026ffdac-484a-40ac-a187-0842d546e8b7 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:40.703638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 578/1154] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:40.710291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922334-99a7-4ad2-a9dc-59eaff3d1a85 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:40.710756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922334-99a7-4ad2-a9dc-59eaff3d1a85 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:40.710848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2922334-99a7-4ad2-a9dc-59eaff3d1a85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:40.734775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a2922334-99a7-4ad2-a9dc-59eaff3d1a85 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:40.736010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a2922334-99a7-4ad2-a9dc-59eaff3d1a85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:40.778635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:40.815732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 577/1155] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:40.839297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ffdac-484a-40ac-a187-0842d546e8b7 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:40.949303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026ffdac-484a-40ac-a187-0842d546e8b7 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:40.983951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dbbfe-ec1f-4da9-95a4-1761e1568e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:40.984664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dbbfe-ec1f-4da9-95a4-1761e1568e0f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:40.984943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dbbfe-ec1f-4da9-95a4-1761e1568e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:40.985220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5dbbfe-ec1f-4da9-95a4-1761e1568e0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:40.996346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5dbbfe-ec1f-4da9-95a4-1761e1568e0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8c5dbbfe-ec1f-4da9-95a4-1761e1568e0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:41.003865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:41.004850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:41.005710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 579/1156] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:01:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:41.013315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2922334-99a7-4ad2-a9dc-59eaff3d1a85 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:41.119079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2922334-99a7-4ad2-a9dc-59eaff3d1a85 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:41.168965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:41.169943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:41.170311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:41.170669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:41.171896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:41.172303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:41.174503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:41.184840 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 578/1157] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:41.190723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dbbfe-ec1f-4da9-95a4-1761e1568e0f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:41.257467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:41.303767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5dbbfe-ec1f-4da9-95a4-1761e1568e0f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:41.348562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9aff4-8db5-4ed7-bb87-b57560c15dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:41.349420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9aff4-8db5-4ed7-bb87-b57560c15dda None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:41.349701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9aff4-8db5-4ed7-bb87-b57560c15dda None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:41.349976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb9aff4-8db5-4ed7-bb87-b57560c15dda None admin] PATH_INFO: `/v3/users/92255253749d405e938df5c35d1a54df` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:41.351340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9aff4-8db5-4ed7-bb87-b57560c15dda None admin] RBAC: Authorizing `identity:get_user(user_id=92255253749d405e938df5c35d1a54df)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:41.352777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb9aff4-8db5-4ed7-bb87-b57560c15dda None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:41.354416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 580/1158] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:41 2022] GET /identity/v3/users/92255253749d405e938df5c35d1a54df => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:41.361111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:41.401055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:41.474274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba19364c-cee5-4d67-af1c-3dcdfc88a70e None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:41.513236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9f8f0-1a96-4a5a-a0c6-0fd48292c447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:41.513834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9f8f0-1a96-4a5a-a0c6-0fd48292c447 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:41.514126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9f8f0-1a96-4a5a-a0c6-0fd48292c447 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:41.514403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9f8f0-1a96-4a5a-a0c6-0fd48292c447 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:41.583754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9f8f0-1a96-4a5a-a0c6-0fd48292c447 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:41.585227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c9f8f0-1a96-4a5a-a0c6-0fd48292c447 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:41.618316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:41.694902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 579/1159] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:41.701405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9aff4-8db5-4ed7-bb87-b57560c15dda None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:41.805775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb9aff4-8db5-4ed7-bb87-b57560c15dda None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:41.830871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:41.842395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859324a-257c-4fad-8c4b-b01c6ac9426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:41.843250 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859324a-257c-4fad-8c4b-b01c6ac9426c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:41.843563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859324a-257c-4fad-8c4b-b01c6ac9426c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:41.843835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859324a-257c-4fad-8c4b-b01c6ac9426c None admin] PATH_INFO: `/v3/projects/25991028b0af4028a346dd89eaf22702` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:41.845055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859324a-257c-4fad-8c4b-b01c6ac9426c None admin] RBAC: Authorizing `identity:get_project(project_id=25991028b0af4028a346dd89eaf22702)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:41.846486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a859324a-257c-4fad-8c4b-b01c6ac9426c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:41.848212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 581/1160] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:41 2022] GET /identity/v3/projects/25991028b0af4028a346dd89eaf22702 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:41.855248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9f8f0-1a96-4a5a-a0c6-0fd48292c447 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:41.957822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9f8f0-1a96-4a5a-a0c6-0fd48292c447 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:41.987914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:41.994117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88183337-34c8-46a3-8173-3a442e60c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:41.994768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183337-34c8-46a3-8173-3a442e60c2bf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:41.995041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183337-34c8-46a3-8173-3a442e60c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:41.995323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88183337-34c8-46a3-8173-3a442e60c2bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:41.996154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88183337-34c8-46a3-8173-3a442e60c2bf None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=25991028b0af4028a346dd89eaf22702, user.id=92255253749d405e938df5c35d1a54df) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:41.996460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88183337-34c8-46a3-8173-3a442e60c2bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:41.997898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88183337-34c8-46a3-8173-3a442e60c2bf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:42.003875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 580/1161] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:41 2022] GET /identity/v3/role_assignments?user.id=92255253749d405e938df5c35d1a54df&scope.project.id=25991028b0af4028a346dd89eaf22702&role.id=6dca675ade814980a252f800fb4fa3ad => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:44.188978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b035475-ccc5-4c9a-96a3-ca337d5c2e0a None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:44.189847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b035475-ccc5-4c9a-96a3-ca337d5c2e0a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:44.189847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b035475-ccc5-4c9a-96a3-ca337d5c2e0a None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:44.191925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 582/1162] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:44.198264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113758-10c8-4f95-8c60-2fffd3c01e41 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:44.198985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113758-10c8-4f95-8c60-2fffd3c01e41 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:44.199137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113758-10c8-4f95-8c60-2fffd3c01e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:44.220889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-5b113758-10c8-4f95-8c60-2fffd3c01e41 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:44.222062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5b113758-10c8-4f95-8c60-2fffd3c01e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:44.293509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 581/1163] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:44.325972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b035475-ccc5-4c9a-96a3-ca337d5c2e0a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:44.439429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b035475-ccc5-4c9a-96a3-ca337d5c2e0a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:44.474788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4831b6-01e4-4f07-83e7-da5517949c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:44.475463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4831b6-01e4-4f07-83e7-da5517949c7e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:44.475743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4831b6-01e4-4f07-83e7-da5517949c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:44.476066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4831b6-01e4-4f07-83e7-da5517949c7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:44.544233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:44.552124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4831b6-01e4-4f07-83e7-da5517949c7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:44.553571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4831b6-01e4-4f07-83e7-da5517949c7e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:44.653160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 583/1164] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:01:44.659413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b113758-10c8-4f95-8c60-2fffd3c01e41 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:44.766247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b113758-10c8-4f95-8c60-2fffd3c01e41 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:44.793240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:44.803824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b476ba-8fca-4471-8a93-ac6fd434afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:44.804694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b476ba-8fca-4471-8a93-ac6fd434afcd None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:44.804981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b476ba-8fca-4471-8a93-ac6fd434afcd None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:44.805255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b476ba-8fca-4471-8a93-ac6fd434afcd None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:44.812098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b476ba-8fca-4471-8a93-ac6fd434afcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:44.813572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b476ba-8fca-4471-8a93-ac6fd434afcd None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-d4b476ba-8fca-4471-8a93-ac6fd434afcd None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:44.827817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:44.829016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:01:44.830197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 582/1165] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:01:44 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:44.838861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4831b6-01e4-4f07-83e7-da5517949c7e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:44.935212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:44.942481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4831b6-01e4-4f07-83e7-da5517949c7e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:44.976742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b3336-9de8-484d-be7b-726dd70d8297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:44.977385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b3336-9de8-484d-be7b-726dd70d8297 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:44.977668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b3336-9de8-484d-be7b-726dd70d8297 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:44.977937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b3336-9de8-484d-be7b-726dd70d8297 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:44.978659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b3336-9de8-484d-be7b-726dd70d8297 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:44.978970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b3336-9de8-484d-be7b-726dd70d8297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:44.980376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b3336-9de8-484d-be7b-726dd70d8297 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:44.987615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 584/1166] 10.222.0.69 () {56 vars in 1115 bytes} [Fri May 20 09:01:44 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:47.169591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1ee0b-9c70-465e-a0c1-53718baec0f5 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:47.170439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1ee0b-9c70-465e-a0c1-53718baec0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:47.170439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1ee0b-9c70-465e-a0c1-53718baec0f5 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:47.172710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 583/1167] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:47.178174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc4f6-e0be-420c-bb97-c5613306e103 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:47.178613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc4f6-e0be-420c-bb97-c5613306e103 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:47.178716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62fc4f6-e0be-420c-bb97-c5613306e103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:47.198143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a62fc4f6-e0be-420c-bb97-c5613306e103 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:47.199310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a62fc4f6-e0be-420c-bb97-c5613306e103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:47.280001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 585/1168] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:47.303456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1ee0b-9c70-465e-a0c1-53718baec0f5 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:47.412128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1ee0b-9c70-465e-a0c1-53718baec0f5 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:47.451225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8024c-3e04-4b2b-9f77-fe3c8cf417fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:47.451860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8024c-3e04-4b2b-9f77-fe3c8cf417fc None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:47.452145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8024c-3e04-4b2b-9f77-fe3c8cf417fc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:47.452427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8024c-3e04-4b2b-9f77-fe3c8cf417fc None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:47.453177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8024c-3e04-4b2b-9f77-fe3c8cf417fc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:47.454571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f8024c-3e04-4b2b-9f77-fe3c8cf417fc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:47.472524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 584/1169] 10.222.0.69 () {60 vars in 1138 bytes} [Fri May 20 09:01:47 2022] POST /identity/v3/domains => generated 267 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:49.696110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc004f-0737-4897-ba62-c15ce0e38526 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:49.696669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc004f-0737-4897-ba62-c15ce0e38526 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:49.696861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc004f-0737-4897-ba62-c15ce0e38526 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:49.700265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 586/1170] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:49.709165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ec083-9b2c-4b46-892a-5bc9bb339f31 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:49.709571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ec083-9b2c-4b46-892a-5bc9bb339f31 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:49.709791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0ec083-9b2c-4b46-892a-5bc9bb339f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:49.733124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-5a0ec083-9b2c-4b46-892a-5bc9bb339f31 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:49.734319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5a0ec083-9b2c-4b46-892a-5bc9bb339f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:49.778227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:49.804625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 585/1171] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:49.828495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc004f-0737-4897-ba62-c15ce0e38526 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:49.930441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:49.944310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc004f-0737-4897-ba62-c15ce0e38526 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:49.978671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c4a75c-0c3a-490b-940e-ea9e9ba2919b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:49.979492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a75c-0c3a-490b-940e-ea9e9ba2919b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:49.979779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a75c-0c3a-490b-940e-ea9e9ba2919b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:49.980054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c4a75c-0c3a-490b-940e-ea9e9ba2919b None admin] PATH_INFO: `/v3/domains/4b29df82d5f94091b66923e3cf6fc758` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:49.990688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4a75c-0c3a-490b-940e-ea9e9ba2919b None admin] RBAC: Authorizing `identity:get_domain(domain_id=4b29df82d5f94091b66923e3cf6fc758)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:49.992232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c4a75c-0c3a-490b-940e-ea9e9ba2919b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:49.993874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 587/1172] 10.222.0.69 () {56 vars in 1183 bytes} [Fri May 20 09:01:49 2022] GET /identity/v3/domains/4b29df82d5f94091b66923e3cf6fc758 => generated 267 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:50.000155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ec083-9b2c-4b46-892a-5bc9bb339f31 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:50.109510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0ec083-9b2c-4b46-892a-5bc9bb339f31 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:50.155385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812950-ae81-4d3c-a932-74609ed988ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:50.156042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812950-ae81-4d3c-a932-74609ed988ed None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:50.156327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812950-ae81-4d3c-a932-74609ed988ed None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:50.156600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812950-ae81-4d3c-a932-74609ed988ed None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:50.157314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50812950-ae81-4d3c-a932-74609ed988ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:50.158763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50812950-ae81-4d3c-a932-74609ed988ed None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:50.187230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 586/1173] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:01:49 2022] POST /identity/v3/projects => generated 361 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:52.395761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca8068-52d8-48ed-9fea-15be9b4ab6e0 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:52.396349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca8068-52d8-48ed-9fea-15be9b4ab6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:52.396349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca8068-52d8-48ed-9fea-15be9b4ab6e0 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:52.397729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 588/1174] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:52.404222 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0ffca-5645-48f7-94be-e51d8be4b7e6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:52.404913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0ffca-5645-48f7-94be-e51d8be4b7e6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:52.405085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0ffca-5645-48f7-94be-e51d8be4b7e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:52.432161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-85c0ffca-5645-48f7-94be-e51d8be4b7e6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:52.433339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-85c0ffca-5645-48f7-94be-e51d8be4b7e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:52.457921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:52.503634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 587/1175] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:52.528633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca8068-52d8-48ed-9fea-15be9b4ab6e0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:52.644739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca8068-52d8-48ed-9fea-15be9b4ab6e0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:52.680050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd21d88-8e7e-4431-b48e-2b5dbd842e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:52.680823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd21d88-8e7e-4431-b48e-2b5dbd842e4b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:52.681103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd21d88-8e7e-4431-b48e-2b5dbd842e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:52.681367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd21d88-8e7e-4431-b48e-2b5dbd842e4b None admin] PATH_INFO: `/v3/domains/4b29df82d5f94091b66923e3cf6fc758` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:52.682632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd21d88-8e7e-4431-b48e-2b5dbd842e4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=4b29df82d5f94091b66923e3cf6fc758)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:52.684089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd21d88-8e7e-4431-b48e-2b5dbd842e4b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:52.685666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 589/1176] 10.222.0.69 () {56 vars in 1183 bytes} [Fri May 20 09:01:52 2022] GET /identity/v3/domains/4b29df82d5f94091b66923e3cf6fc758 => generated 267 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:52.692119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0ffca-5645-48f7-94be-e51d8be4b7e6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:52.797584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0ffca-5645-48f7-94be-e51d8be4b7e6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:52.832704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2655aa15-74b1-4395-b264-75b016204781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:52.833360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655aa15-74b1-4395-b264-75b016204781 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:52.833636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655aa15-74b1-4395-b264-75b016204781 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:52.833906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2655aa15-74b1-4395-b264-75b016204781 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:52.834614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655aa15-74b1-4395-b264-75b016204781 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:52.836145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2655aa15-74b1-4395-b264-75b016204781 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:52.861483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 588/1177] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 09:01:52 2022] POST /identity/v3/users => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:01:55.103860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a196-92f8-499a-9834-13e324cd9213 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:55.104407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a196-92f8-499a-9834-13e324cd9213 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:55.104407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a196-92f8-499a-9834-13e324cd9213 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:55.105664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 590/1178] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:55.112098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6c48-53b4-477c-a130-e3595dcc1c47 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:55.112856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6c48-53b4-477c-a130-e3595dcc1c47 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:55.112999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6a6c48-53b4-477c-a130-e3595dcc1c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:55.131650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0d6a6c48-53b4-477c-a130-e3595dcc1c47 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:55.132836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0d6a6c48-53b4-477c-a130-e3595dcc1c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:55.141980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:55.203447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 589/1179] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:55.226595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a196-92f8-499a-9834-13e324cd9213 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:55.245456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:55.331881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a196-92f8-499a-9834-13e324cd9213 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:55.367033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfebf4-eae9-4f54-955a-cff9b59c1a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:55.367766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfebf4-eae9-4f54-955a-cff9b59c1a0c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:55.368027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfebf4-eae9-4f54-955a-cff9b59c1a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:55.368293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfebf4-eae9-4f54-955a-cff9b59c1a0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:55.378048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfebf4-eae9-4f54-955a-cff9b59c1a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-3ccfebf4-eae9-4f54-955a-cff9b59c1a0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:55.385576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:55.386662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:55.387599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 591/1180] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:01:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:55.395420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a6c48-53b4-477c-a130-e3595dcc1c47 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:55.497696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6a6c48-53b4-477c-a130-e3595dcc1c47 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:55.533830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:55.534483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:55.534761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:55.535026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:55.535807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:55.536111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:55.537492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:55.545918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 590/1181] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:01:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:55.552572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfebf4-eae9-4f54-955a-cff9b59c1a0c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:55.638334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:55.669800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfebf4-eae9-4f54-955a-cff9b59c1a0c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:55.704752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de8e7f-a358-4379-ace0-13c907ffdba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:55.705616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de8e7f-a358-4379-ace0-13c907ffdba1 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:55.705899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de8e7f-a358-4379-ace0-13c907ffdba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:55.706165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3de8e7f-a358-4379-ace0-13c907ffdba1 None admin] PATH_INFO: `/v3/users/819b727d2a8744b1891cca0c2ad69fd7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:55.719695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de8e7f-a358-4379-ace0-13c907ffdba1 None admin] RBAC: Authorizing `identity:get_user(user_id=819b727d2a8744b1891cca0c2ad69fd7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:55.721122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3de8e7f-a358-4379-ace0-13c907ffdba1 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:55.722715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 592/1182] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:55 2022] GET /identity/v3/users/819b727d2a8744b1891cca0c2ad69fd7 => generated 311 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:55.729166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:55.739110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:55.835090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec29a4-16a3-4534-a6ee-2baaad8b8285 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:55.868934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4f42b-8eea-4a11-8009-5f3a6f13e866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:55.869563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f42b-8eea-4a11-8009-5f3a6f13e866 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:55.869841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f42b-8eea-4a11-8009-5f3a6f13e866 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:55.870109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4f42b-8eea-4a11-8009-5f3a6f13e866 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:55.941578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4f42b-8eea-4a11-8009-5f3a6f13e866 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:55.942954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda4f42b-8eea-4a11-8009-5f3a6f13e866 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:55.950682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:56.050907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 591/1183] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:01:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 20 09:01:56.058321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de8e7f-a358-4379-ace0-13c907ffdba1 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:56.163223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3de8e7f-a358-4379-ace0-13c907ffdba1 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:56.198533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7314cc-34b5-49d0-8a19-d4b730971652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:56.199304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7314cc-34b5-49d0-8a19-d4b730971652 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:56.199614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7314cc-34b5-49d0-8a19-d4b730971652 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:56.199882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7314cc-34b5-49d0-8a19-d4b730971652 None admin] PATH_INFO: `/v3/projects/101be10871b0489fabcd903a135f2027` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:56.201193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7314cc-34b5-49d0-8a19-d4b730971652 None admin] RBAC: Authorizing `identity:get_project(project_id=101be10871b0489fabcd903a135f2027)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:56.202621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7314cc-34b5-49d0-8a19-d4b730971652 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:56.204338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 593/1184] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:56 2022] GET /identity/v3/projects/101be10871b0489fabcd903a135f2027 => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:56.210574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4f42b-8eea-4a11-8009-5f3a6f13e866 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:56.313538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4f42b-8eea-4a11-8009-5f3a6f13e866 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:56.320898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:56.354203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011fbad-7f91-40a9-b2f2-7fad4daa99d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:56.354850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011fbad-7f91-40a9-b2f2-7fad4daa99d1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:56.355134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011fbad-7f91-40a9-b2f2-7fad4daa99d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:56.355438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011fbad-7f91-40a9-b2f2-7fad4daa99d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:56.356277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011fbad-7f91-40a9-b2f2-7fad4daa99d1 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=101be10871b0489fabcd903a135f2027, user.id=819b727d2a8744b1891cca0c2ad69fd7) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:56.356588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011fbad-7f91-40a9-b2f2-7fad4daa99d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:56.358043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011fbad-7f91-40a9-b2f2-7fad4daa99d1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:56.364058 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 592/1185] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:01:56 2022] GET /identity/v3/role_assignments?user.id=819b727d2a8744b1891cca0c2ad69fd7&scope.project.id=101be10871b0489fabcd903a135f2027&role.id=6dca675ade814980a252f800fb4fa3ad => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:58.506854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a725f33-5dcc-4411-ad35-c51e5c6d780d None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:58.507448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a725f33-5dcc-4411-ad35-c51e5c6d780d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:58.507448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a725f33-5dcc-4411-ad35-c51e5c6d780d None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:58.508917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 594/1186] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:01:58.515569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc7860-8d6f-45b5-ad01-94addb341df6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:58.516284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc7860-8d6f-45b5-ad01-94addb341df6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:58.516416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc7860-8d6f-45b5-ad01-94addb341df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:58.543707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-efbc7860-8d6f-45b5-ad01-94addb341df6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:01:58.544895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-efbc7860-8d6f-45b5-ad01-94addb341df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:01:58.611526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 593/1187] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:01:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:01:58.642527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a725f33-5dcc-4411-ad35-c51e5c6d780d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:58.645619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:58.748883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a725f33-5dcc-4411-ad35-c51e5c6d780d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:58.783351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486f17-ffa0-4809-aaf4-465a17ae69fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:58.784138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486f17-ffa0-4809-aaf4-465a17ae69fc None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:58.784415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486f17-ffa0-4809-aaf4-465a17ae69fc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:58.784684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd486f17-ffa0-4809-aaf4-465a17ae69fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:58.795637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd486f17-ffa0-4809-aaf4-465a17ae69fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-dd486f17-ffa0-4809-aaf4-465a17ae69fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:01:58.803060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:01:58.804152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:01:58.805136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 595/1188] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:01:58.810484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc7860-8d6f-45b5-ad01-94addb341df6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:58.919480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:58.922795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc7860-8d6f-45b5-ad01-94addb341df6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:58.958441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:58.959097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:58.959409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:58.959686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:58.960473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:01:58.960778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:58.962210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:58.970780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 594/1189] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:01:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:58.977308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486f17-ffa0-4809-aaf4-465a17ae69fc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:59.060774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:59.099792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd486f17-ffa0-4809-aaf4-465a17ae69fc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:59.137535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704601c-9b8d-4b9a-9144-4dcf83998317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:59.138391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704601c-9b8d-4b9a-9144-4dcf83998317 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:59.138673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704601c-9b8d-4b9a-9144-4dcf83998317 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:59.138940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5704601c-9b8d-4b9a-9144-4dcf83998317 None admin] PATH_INFO: `/v3/users/819b727d2a8744b1891cca0c2ad69fd7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:59.140293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704601c-9b8d-4b9a-9144-4dcf83998317 None admin] RBAC: Authorizing `identity:get_user(user_id=819b727d2a8744b1891cca0c2ad69fd7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:59.141790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5704601c-9b8d-4b9a-9144-4dcf83998317 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:59.143422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 596/1190] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:01:58 2022] GET /identity/v3/users/819b727d2a8744b1891cca0c2ad69fd7 => generated 311 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:59.151170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:59.255814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba74641d-f00e-46c6-a2f8-d7117ce3cadf None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:59.288983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7780156-c4a9-4834-9ce8-5fda161789fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:59.289845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7780156-c4a9-4834-9ce8-5fda161789fa None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:59.290126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7780156-c4a9-4834-9ce8-5fda161789fa None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:59.290395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7780156-c4a9-4834-9ce8-5fda161789fa None admin] PATH_INFO: `/v3/projects/101be10871b0489fabcd903a135f2027` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:59.291602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7780156-c4a9-4834-9ce8-5fda161789fa None admin] RBAC: Authorizing `identity:get_project(project_id=101be10871b0489fabcd903a135f2027)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:59.293040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7780156-c4a9-4834-9ce8-5fda161789fa None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:59.294736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 595/1191] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:01:59 2022] GET /identity/v3/projects/101be10871b0489fabcd903a135f2027 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:01:59.300745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704601c-9b8d-4b9a-9144-4dcf83998317 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:01:59.398695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5704601c-9b8d-4b9a-9144-4dcf83998317 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:01:59.426336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:01:59.432475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f47608-b157-40e1-bafd-515afb02be6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:01:59.433365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f47608-b157-40e1-bafd-515afb02be6b None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:01:59.433647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f47608-b157-40e1-bafd-515afb02be6b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:01:59.433923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f47608-b157-40e1-bafd-515afb02be6b None admin] PATH_INFO: `/v3/projects/101be10871b0489fabcd903a135f2027/users/819b727d2a8744b1891cca0c2ad69fd7/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:01:59.435918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f47608-b157-40e1-bafd-515afb02be6b None admin] RBAC: Authorizing `identity:create_grant(project_id=101be10871b0489fabcd903a135f2027, user_id=819b727d2a8744b1891cca0c2ad69fd7, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:01:59.437371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f47608-b157-40e1-bafd-515afb02be6b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:01:59.453798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 597/1192] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:01:59 2022] PUT /identity/v3/projects/101be10871b0489fabcd903a135f2027/users/819b727d2a8744b1891cca0c2ad69fd7/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:02:01.601232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b08f8-9ed7-44f9-b7f4-ce6c385abce8 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:02:01.601898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b08f8-9ed7-44f9-b7f4-ce6c385abce8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:02:01.601898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b08f8-9ed7-44f9-b7f4-ce6c385abce8 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:02:01.603939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 596/1193] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:02:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:02:01.612965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d1708-2191-4484-bb0b-8ca87e581078 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:02:01.613353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d1708-2191-4484-bb0b-8ca87e581078 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:02:01.613593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133d1708-2191-4484-bb0b-8ca87e581078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:02:01.638237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-133d1708-2191-4484-bb0b-8ca87e581078 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:02:01.639450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-133d1708-2191-4484-bb0b-8ca87e581078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:02:01.715406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 598/1194] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:02:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:02:01.742906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b08f8-9ed7-44f9-b7f4-ce6c385abce8 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:02:01.778007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:02:01.855595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b08f8-9ed7-44f9-b7f4-ce6c385abce8 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:02:01.899154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717761f-7498-4b19-8446-eff7f1725c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:02:01.899982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717761f-7498-4b19-8446-eff7f1725c31 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:02:01.900259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717761f-7498-4b19-8446-eff7f1725c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:02:01.900532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717761f-7498-4b19-8446-eff7f1725c31 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:02:01.911413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7717761f-7498-4b19-8446-eff7f1725c31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-7717761f-7498-4b19-8446-eff7f1725c31 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:02:01.919302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:02:01.920382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:02:01.921847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 597/1195] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:02:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:02:01.929873 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d1708-2191-4484-bb0b-8ca87e581078 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:02:02.029181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133d1708-2191-4484-bb0b-8ca87e581078 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:02:02.051821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:02:02.068664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:02:02.069329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:02:02.069608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:02:02.069891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:02:02.070648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:02:02.070951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:02:02.072361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:02:02.080647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 599/1196] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:02:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:02:02.086760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717761f-7498-4b19-8446-eff7f1725c31 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:02:02.189511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:02:02.199890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7717761f-7498-4b19-8446-eff7f1725c31 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:02:02.237893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6aad9c-0f2a-4cf4-8227-2484c9a76efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:02:02.238780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6aad9c-0f2a-4cf4-8227-2484c9a76efb None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:02:02.239061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6aad9c-0f2a-4cf4-8227-2484c9a76efb None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:02:02.239351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6aad9c-0f2a-4cf4-8227-2484c9a76efb None admin] PATH_INFO: `/v3/users/819b727d2a8744b1891cca0c2ad69fd7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:02:02.240550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6aad9c-0f2a-4cf4-8227-2484c9a76efb None admin] RBAC: Authorizing `identity:get_user(user_id=819b727d2a8744b1891cca0c2ad69fd7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:02:02.242006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6aad9c-0f2a-4cf4-8227-2484c9a76efb None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:02:02.243692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 598/1197] 10.222.0.69 () {56 vars in 1177 bytes} [Fri May 20 09:02:02 2022] GET /identity/v3/users/819b727d2a8744b1891cca0c2ad69fd7 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:02:02.252732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:02:02.356922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db672036-dd6a-497a-b265-7533ba17e04c None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:02:02.391335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed06212-ee2d-4c44-9a1e-524817f8de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:02:02.391966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed06212-ee2d-4c44-9a1e-524817f8de29 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:02:02.392248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed06212-ee2d-4c44-9a1e-524817f8de29 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:02:02.392520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed06212-ee2d-4c44-9a1e-524817f8de29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:02:02.441745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:02:02.462419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed06212-ee2d-4c44-9a1e-524817f8de29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:02:02.463900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed06212-ee2d-4c44-9a1e-524817f8de29 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:02:02.572815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 600/1198] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:02:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:02:02.579661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6aad9c-0f2a-4cf4-8227-2484c9a76efb None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:02:02.693124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6aad9c-0f2a-4cf4-8227-2484c9a76efb None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:02:02.727437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8752b4b5-7108-4448-9ab0-07379b2fce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:02:02.728293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b4b5-7108-4448-9ab0-07379b2fce05 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:02:02.728579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b4b5-7108-4448-9ab0-07379b2fce05 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:02:02.728853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8752b4b5-7108-4448-9ab0-07379b2fce05 None admin] PATH_INFO: `/v3/projects/101be10871b0489fabcd903a135f2027` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:02:02.730155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b4b5-7108-4448-9ab0-07379b2fce05 None admin] RBAC: Authorizing `identity:get_project(project_id=101be10871b0489fabcd903a135f2027)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:02:02.731649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8752b4b5-7108-4448-9ab0-07379b2fce05 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:02:02.733333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 599/1199] 10.222.0.69 () {56 vars in 1186 bytes} [Fri May 20 09:02:02 2022] GET /identity/v3/projects/101be10871b0489fabcd903a135f2027 => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:02:02.739451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed06212-ee2d-4c44-9a1e-524817f8de29 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:02:02.828245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:02:02.850308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed06212-ee2d-4c44-9a1e-524817f8de29 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:02:02.884434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd72755-e9db-44b2-b3ff-e4aefe2fb617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:02:02.885067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72755-e9db-44b2-b3ff-e4aefe2fb617 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:02:02.885347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72755-e9db-44b2-b3ff-e4aefe2fb617 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:02:02.885611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd72755-e9db-44b2-b3ff-e4aefe2fb617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:02:02.886436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd72755-e9db-44b2-b3ff-e4aefe2fb617 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=101be10871b0489fabcd903a135f2027, user.id=819b727d2a8744b1891cca0c2ad69fd7) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:02:02.886743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd72755-e9db-44b2-b3ff-e4aefe2fb617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:02:02.888193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd72755-e9db-44b2-b3ff-e4aefe2fb617 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:02:02.894046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 601/1200] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:02:02 2022] GET /identity/v3/role_assignments?user.id=819b727d2a8744b1891cca0c2ad69fd7&scope.project.id=101be10871b0489fabcd903a135f2027&role.id=6dca675ade814980a252f800fb4fa3ad => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:02:12.722458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=87 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:02:12.891364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=27 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:03.571294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177a4d2-6038-4ccc-9995-d3c80868680a None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:03.571294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177a4d2-6038-4ccc-9995-d3c80868680a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:03.571294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6177a4d2-6038-4ccc-9995-d3c80868680a None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:03.571294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 600/1201] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:03.579407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67f97d-2225-4afd-be47-ef35db5f5852 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:03.579407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67f97d-2225-4afd-be47-ef35db5f5852 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:03.579407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd67f97d-2225-4afd-be47-ef35db5f5852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:03.597159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-dd67f97d-2225-4afd-be47-ef35db5f5852 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:03.598433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-dd67f97d-2225-4afd-be47-ef35db5f5852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:03.691314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 602/1202] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:03:03.718413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177a4d2-6038-4ccc-9995-d3c80868680a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:03.839088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6177a4d2-6038-4ccc-9995-d3c80868680a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:03.881832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:03.882713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:03.883047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:03.883373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:03.895422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:03.896919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:03.917877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:03.919418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:03.920629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:03.920629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:03.920629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:03.920629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:03.922622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 601/1203] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:03.928901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67f97d-2225-4afd-be47-ef35db5f5852 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:04.066310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd67f97d-2225-4afd-be47-ef35db5f5852 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:04.101083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:04.108700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9156def2-6f6d-428c-8cae-e594b678ad07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:04.109419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156def2-6f6d-428c-8cae-e594b678ad07 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:04.109700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156def2-6f6d-428c-8cae-e594b678ad07 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:04.109977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9156def2-6f6d-428c-8cae-e594b678ad07 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:04.110668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156def2-6f6d-428c-8cae-e594b678ad07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:04.110973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156def2-6f6d-428c-8cae-e594b678ad07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:04.112548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9156def2-6f6d-428c-8cae-e594b678ad07 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:04.124361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 603/1204] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:04.130564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:04.196816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:04.257946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a42a4a-64fb-461b-9349-3017353328c6 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:04.298218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2f033d-d778-44cb-9498-acc736f0f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:04.298926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f033d-d778-44cb-9498-acc736f0f629 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:04.299406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f033d-d778-44cb-9498-acc736f0f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:04.299668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2f033d-d778-44cb-9498-acc736f0f629 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:04.304839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f033d-d778-44cb-9498-acc736f0f629 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:04.304839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2f033d-d778-44cb-9498-acc736f0f629 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:04.326574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 602/1205] 10.222.0.69 () {60 vars in 1132 bytes} [Fri May 20 09:03:04 2022] POST /identity/v3/users => generated 251 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:03:06.726905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaa412-1840-4b5b-85cf-23de6a0bb3b8 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:06.727540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaa412-1840-4b5b-85cf-23de6a0bb3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:06.727540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaa412-1840-4b5b-85cf-23de6a0bb3b8 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:06.728874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 604/1206] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:06.734692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de668b-c596-44dc-a625-fa272d574553 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:06.735303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de668b-c596-44dc-a625-fa272d574553 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:06.735303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9de668b-c596-44dc-a625-fa272d574553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:06.752332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f9de668b-c596-44dc-a625-fa272d574553 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:06.753570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f9de668b-c596-44dc-a625-fa272d574553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:06.841602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 603/1207] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:06.865161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aaa412-1840-4b5b-85cf-23de6a0bb3b8 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:06.983194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aaa412-1840-4b5b-85cf-23de6a0bb3b8 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:07.020230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2111f3-7a7b-4c38-92ff-a5eb7ec8bb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:07.021034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2111f3-7a7b-4c38-92ff-a5eb7ec8bb34 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:07.021311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2111f3-7a7b-4c38-92ff-a5eb7ec8bb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:07.021613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2111f3-7a7b-4c38-92ff-a5eb7ec8bb34 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:07.032527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2111f3-7a7b-4c38-92ff-a5eb7ec8bb34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-7d2111f3-7a7b-4c38-92ff-a5eb7ec8bb34 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:03:07.041081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:07.042341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:07.043848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 605/1208] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:03:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:03:07.048558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de668b-c596-44dc-a625-fa272d574553 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:07.093939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:07.156175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9de668b-c596-44dc-a625-fa272d574553 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:07.191986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:07.192661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:07.192940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:07.193228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:07.193963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:07.194274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:07.195684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:07.207013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 604/1209] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:03:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:07.213256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2111f3-7a7b-4c38-92ff-a5eb7ec8bb34 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:07.274432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:07.330997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2111f3-7a7b-4c38-92ff-a5eb7ec8bb34 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:07.369836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:07.371020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:07.371020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:07.371354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:07.384695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:07.386262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:07.406336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:07.407439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:07.415403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:07.415403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:07.415403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:07.415403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:07.415403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 606/1210] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:07.415735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:07.533350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:07.536674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770c653-efb1-4236-a994-dfd03ed53e92 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:07.575022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:07.575739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:07.576018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:07.576291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:07.577017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:07.577320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:07.578711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:07.590566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 605/1211] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:07.596126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:07.706424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d25dc-a255-433d-b3a9-28e32cef586d None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:07.707977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:07.746686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:07.747672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:07.748010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:07.748277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:07.757703 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:07.759207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:03:07.775294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:07.776977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:07.777962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 607/1212] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:07.781642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:07.892409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebe9437-7e50-4484-a264-3ff5a280a249 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:07.932994 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:07.933671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:07.933945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:07.934214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:07.935054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:07.936711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:07.936887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:07.954344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 606/1213] 10.222.0.69 () {56 vars in 1143 bytes} [Fri May 20 09:03:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:07.960243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:08.073832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574dcad2-2d43-4953-b17d-4ad8cb3f0994 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:08.110798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c903f0-2ba5-4dc3-ad28-b52f189b7a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:08.111581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c903f0-2ba5-4dc3-ad28-b52f189b7a34 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:08.111912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c903f0-2ba5-4dc3-ad28-b52f189b7a34 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:08.112149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c903f0-2ba5-4dc3-ad28-b52f189b7a34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:08.147551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:08.188482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c903f0-2ba5-4dc3-ad28-b52f189b7a34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:08.190249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c903f0-2ba5-4dc3-ad28-b52f189b7a34 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:08.294300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 608/1214] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:03:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 20 09:03:08.300219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:08.323511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:08.418026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ae1f8e-b521-4ce4-99cc-85b21f98996a None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:08.455851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:08.456767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:08.457097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:08.457394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:08.469295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:08.471091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:08.490903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:08.492060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:08.493343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:08.493343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:08.493343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:08.493343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:08.493343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 607/1215] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:08.499738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c903f0-2ba5-4dc3-ad28-b52f189b7a34 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:08.572907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:08.615171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c903f0-2ba5-4dc3-ad28-b52f189b7a34 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:08.653066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:08.653733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:08.654017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:08.654293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:08.655054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:08.655390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:08.656785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:08.670455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 609/1216] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:08.674817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:08.756915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:08.791859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037de54f-a998-425e-88f5-f1cd0faf58a6 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:08.829410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:08.830506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:08.830929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:08.831394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:08.839638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:08.841455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:03:08.858526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:08.859757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:08.860693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 608/1217] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:03:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:08.865779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:08.971844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8972e5-b513-4587-af30-08594de14c3f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:08.993874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:09.006787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7c68f4-1844-4f31-9b64-ad18c18dc5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:09.007475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c68f4-1844-4f31-9b64-ad18c18dc5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:09.007812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c68f4-1844-4f31-9b64-ad18c18dc5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:09.008027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7c68f4-1844-4f31-9b64-ad18c18dc5b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:09.008818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c68f4-1844-4f31-9b64-ad18c18dc5b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:09.009129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c68f4-1844-4f31-9b64-ad18c18dc5b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:09.010589 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7c68f4-1844-4f31-9b64-ad18c18dc5b2 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:09.028310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 610/1218] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:03:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:09.029191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:09.136199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1440af64-fbc2-42ca-a973-4d6a735de7b8 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:09.173696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:09.180427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded2543-3995-4ced-990f-4c9b2f65d7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:09.181200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2543-3995-4ced-990f-4c9b2f65d7ad None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:09.181580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2543-3995-4ced-990f-4c9b2f65d7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:09.182003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2543-3995-4ced-990f-4c9b2f65d7ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:09.182959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded2543-3995-4ced-990f-4c9b2f65d7ad None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=6def7fd117dc4b7598a1864e63f09427) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:09.183412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded2543-3995-4ced-990f-4c9b2f65d7ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:09.184955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded2543-3995-4ced-990f-4c9b2f65d7ad None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:09.192204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 609/1219] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:03:09 2022] GET /identity/v3/role_assignments?user.id=6def7fd117dc4b7598a1864e63f09427&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:11.391799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d429dc-547b-4da3-8828-13d26af051db None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:11.392406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d429dc-547b-4da3-8828-13d26af051db None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:11.392406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d429dc-547b-4da3-8828-13d26af051db None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:11.393974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 611/1220] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:11.400106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde549f5-118c-4ac4-b7dc-e27ad0176c7f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:11.400698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde549f5-118c-4ac4-b7dc-e27ad0176c7f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:11.400698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde549f5-118c-4ac4-b7dc-e27ad0176c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:11.420028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-bde549f5-118c-4ac4-b7dc-e27ad0176c7f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:11.421429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bde549f5-118c-4ac4-b7dc-e27ad0176c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:11.507324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 610/1221] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:11.537102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d429dc-547b-4da3-8828-13d26af051db None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:11.659689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d429dc-547b-4da3-8828-13d26af051db None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:11.697471 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa38e34-0409-4b63-9fcf-684213c724cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:11.698273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa38e34-0409-4b63-9fcf-684213c724cc None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:11.698555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa38e34-0409-4b63-9fcf-684213c724cc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:11.698852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa38e34-0409-4b63-9fcf-684213c724cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:11.710297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa38e34-0409-4b63-9fcf-684213c724cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8fa38e34-0409-4b63-9fcf-684213c724cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:03:11.717958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:11.719156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:11.720405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 612/1222] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:03:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:11.725386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde549f5-118c-4ac4-b7dc-e27ad0176c7f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:11.836431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde549f5-118c-4ac4-b7dc-e27ad0176c7f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:11.861224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:11.873133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:11.873822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:11.874082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:11.874357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:11.875110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] RBAC: Adding query filter params (name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:11.875443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:11.876830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:11.885023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 611/1223] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:03:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:11.891036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa38e34-0409-4b63-9fcf-684213c724cc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:11.938401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:11.998436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa38e34-0409-4b63-9fcf-684213c724cc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:12.034928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:12.035762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:12.036422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:12.036692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:12.049986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:12.051853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:12.070509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:12.071640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:12.072660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:12.072660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:12.072660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:12.072660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:12.072660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 613/1224] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:12.078275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:12.190900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df93b605-2bcd-4ea9-a962-ffdfe9281478 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:12.225253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:12.225920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:12.226201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:12.226474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:12.227206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:12.227543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:12.228982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:12.241091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 612/1225] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:12.246770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:12.340610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:12.347732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f220a99-b98c-4faa-9a47-d17861e428b6 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:12.382500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:12.383322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:12.383608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:12.383890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:12.392079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:12.393530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:03:12.407902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:12.409037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:12.410713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 614/1226] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:03:12.416730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:12.464345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:12.517578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17269533-aa06-4a39-b145-e8fa69935b94 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:12.553790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:12.554484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:12.554745 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:12.555018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:12.555886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:12.556264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:12.557703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:12.575242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 613/1227] 10.222.0.69 () {56 vars in 1143 bytes} [Fri May 20 09:03:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:12.581099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:12.682267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5abbbb-7777-49f2-93b4-e135bd5a2fcf None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:12.716871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:12.717723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:12.718032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:12.718371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:12.729312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:12.730766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:12.748679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:12.749843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:12.750907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:12.750907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:12.750907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:12.750907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:12.750907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 615/1228] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:03:12.756072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:12.858931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fcbba-12fb-44dd-8880-9c342eab29bc None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:12.861333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:12.894422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:12.895197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:12.895543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:12.895859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:12.896703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:12.897072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:12.898629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:12.910445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 614/1229] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:12.916652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:12.925719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:13.032700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53233770-404c-441d-a8f4-b9fa4b0e2ec6 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:13.078653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:13.079559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:13.079864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:13.080141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:13.087971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:13.089913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:03:13.105656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:13.106939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:13.107923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 616/1230] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:03:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:13.113246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:13.221335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f25ed-d1f9-4386-b2a5-2915404b82f4 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:13.257629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58293b-85b0-4706-bd06-6ce6e19e47b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:13.258325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58293b-85b0-4706-bd06-6ce6e19e47b4 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:13.258609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58293b-85b0-4706-bd06-6ce6e19e47b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:13.258886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58293b-85b0-4706-bd06-6ce6e19e47b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:13.259780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58293b-85b0-4706-bd06-6ce6e19e47b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:13.260094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58293b-85b0-4706-bd06-6ce6e19e47b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:13.261565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58293b-85b0-4706-bd06-6ce6e19e47b4 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:13.274204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 615/1231] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:03:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:13.280554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:13.352734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:13.389948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea156705-ac6f-44e1-a880-a723372c7f13 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:13.424925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e21c0d-c190-4d94-bedb-302493947f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:13.425816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21c0d-c190-4d94-bedb-302493947f9b None admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:13.426089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21c0d-c190-4d94-bedb-302493947f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:13.426357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e21c0d-c190-4d94-bedb-302493947f9b None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/6def7fd117dc4b7598a1864e63f09427/roles/99ad2a72e54b4e51baabe132c647c8e3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:13.441115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21c0d-c190-4d94-bedb-302493947f9b None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=6def7fd117dc4b7598a1864e63f09427, role_id=99ad2a72e54b4e51baabe132c647c8e3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:13.442615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e21c0d-c190-4d94-bedb-302493947f9b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:13.455694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 617/1232] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:03:13 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/6def7fd117dc4b7598a1864e63f09427/roles/99ad2a72e54b4e51baabe132c647c8e3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:03:15.714647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cd15f-6066-43a9-9e25-97296d3db8e2 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:15.715293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cd15f-6066-43a9-9e25-97296d3db8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:15.715293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cd15f-6066-43a9-9e25-97296d3db8e2 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:15.716611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 616/1233] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:15.722046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a8d7-861a-4233-845f-4a1aecefbaa0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:15.722474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a8d7-861a-4233-845f-4a1aecefbaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:15.722606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3161a8d7-861a-4233-845f-4a1aecefbaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:15.740553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3161a8d7-861a-4233-845f-4a1aecefbaa0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:15.741843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3161a8d7-861a-4233-845f-4a1aecefbaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:15.813230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 618/1234] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:15.836924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cd15f-6066-43a9-9e25-97296d3db8e2 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:15.862739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:15.940968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cd15f-6066-43a9-9e25-97296d3db8e2 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:15.981197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c96f6-b0c5-454d-8988-9da8ecbe78f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:15.981927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c96f6-b0c5-454d-8988-9da8ecbe78f0 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:15.982210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c96f6-b0c5-454d-8988-9da8ecbe78f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:15.982485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360c96f6-b0c5-454d-8988-9da8ecbe78f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:15.993506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360c96f6-b0c5-454d-8988-9da8ecbe78f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-360c96f6-b0c5-454d-8988-9da8ecbe78f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:03:16.001469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:16.002629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:16.003697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 617/1235] 10.222.0.69 () {56 vars in 1102 bytes} [Fri May 20 09:03:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:16.009656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161a8d7-861a-4233-845f-4a1aecefbaa0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:16.059339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:16.113475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3161a8d7-861a-4233-845f-4a1aecefbaa0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:16.147280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:16.147941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:16.148219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:16.148586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:16.149336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:16.149634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:16.151052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:16.159360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 619/1236] 10.222.0.69 () {56 vars in 1103 bytes} [Fri May 20 09:03:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:16.166840 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c96f6-b0c5-454d-8988-9da8ecbe78f0 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:16.252289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:16.268806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360c96f6-b0c5-454d-8988-9da8ecbe78f0 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:16.307938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:16.308740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:16.309020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:16.309292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:16.320228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:16.321692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:16.338943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:16.340109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:16.341324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:16.341324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:16.341324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:16.341324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:16.341324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 618/1237] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:16.346198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:16.457451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16485c78-f5ea-48ab-a881-f78e776f5a77 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:16.468011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:16.495155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:16.495925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:16.496210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:16.496519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:16.497314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:16.497646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:16.499108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:16.512301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 620/1238] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:16.518359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:16.636434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc52e3-f971-4880-97db-4566af17898d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:16.670387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:16.671328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:16.671681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:16.672024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:16.674705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:16.680227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:16.682083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:03:16.697649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:16.698883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:16.700346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 619/1239] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:16.705373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:16.811737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eecec7-44f1-444d-b3c0-e2ba71ee5241 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:16.848149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:16.848801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:16.849082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:16.849355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:16.850199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:16.850506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:16.851972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:16.869034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 621/1240] 10.222.0.69 () {56 vars in 1143 bytes} [Fri May 20 09:03:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:16.874857 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:16.984833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcb0de-4fe3-4411-8154-55d262e4b303 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:17.024155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23287d97-b231-49ec-b13f-90021aed399d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:17.024758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23287d97-b231-49ec-b13f-90021aed399d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:17.025096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23287d97-b231-49ec-b13f-90021aed399d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:17.025396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23287d97-b231-49ec-b13f-90021aed399d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:17.091909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:17.099166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23287d97-b231-49ec-b13f-90021aed399d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:17.101525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23287d97-b231-49ec-b13f-90021aed399d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:17.203184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 620/1241] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:03:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:03:17.209528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:17.237234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:17.318234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d13e09-3fc2-4251-9e97-990290716b02 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:17.359270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:17.360092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:17.360378 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:17.360655 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:17.371744 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:17.373543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:17.392421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:17.393702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:17.394953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:17.394953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:17.394953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:17.394953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:17.394953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 622/1242] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:03:17.400117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23287d97-b231-49ec-b13f-90021aed399d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:17.495891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:17.503283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23287d97-b231-49ec-b13f-90021aed399d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:17.542261 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:17.542934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:17.543213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:17.543650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:17.544310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:17.544663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:17.545960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:17.558335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 621/1243] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:03:17.565035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:17.650855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:17.674162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0227acc4-49fc-45e9-b973-9ead7c4816f7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:17.709800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:17.710669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:17.710991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:17.711293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:17.718775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:17.720343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:03:17.734516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:17.735502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:17.736517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:03:17.736517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:03:17.736517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:03:17.736517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:03:17.736517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:03:17.736517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:03:17.736517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:17.736517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:17.737017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 623/1244] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:03:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:17.744340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:17.854095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465f933-7a35-40e0-9a87-5c669f58a661 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:17.888251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde88d3f-d729-4965-b489-33d63a224d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:17.888930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88d3f-d729-4965-b489-33d63a224d6d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:17.889199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88d3f-d729-4965-b489-33d63a224d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:17.889470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde88d3f-d729-4965-b489-33d63a224d6d None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:17.890477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde88d3f-d729-4965-b489-33d63a224d6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:17.890797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde88d3f-d729-4965-b489-33d63a224d6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:17.892431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde88d3f-d729-4965-b489-33d63a224d6d None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:17.907781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 622/1245] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:03:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:17.914452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:18.026728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbc9601-bae8-4361-ba7e-210d443ba193 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:18.062422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375a020-d85a-414a-8e49-9007b34e9e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:18.063165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375a020-d85a-414a-8e49-9007b34e9e97 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:18.063508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375a020-d85a-414a-8e49-9007b34e9e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:18.063804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375a020-d85a-414a-8e49-9007b34e9e97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:18.064709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375a020-d85a-414a-8e49-9007b34e9e97 None admin] RBAC: Adding query filter params (role.id=99ad2a72e54b4e51baabe132c647c8e3, scope.project.id=569782752acb4085ad742330af999bc2, user.id=6def7fd117dc4b7598a1864e63f09427) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:18.065080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375a020-d85a-414a-8e49-9007b34e9e97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:18.066560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375a020-d85a-414a-8e49-9007b34e9e97 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:18.073230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 624/1246] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:03:17 2022] GET /identity/v3/role_assignments?user.id=6def7fd117dc4b7598a1864e63f09427&scope.project.id=569782752acb4085ad742330af999bc2&role.id=99ad2a72e54b4e51baabe132c647c8e3 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:20.303173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec03f-47d7-4293-8d7c-2bf3786ef1fc None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:20.303781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec03f-47d7-4293-8d7c-2bf3786ef1fc None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:20.303781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ec03f-47d7-4293-8d7c-2bf3786ef1fc None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:20.305540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 623/1247] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:20.312794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a75b0-6fc0-4512-bd9c-7e729d0d69aa None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:20.312794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a75b0-6fc0-4512-bd9c-7e729d0d69aa None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:20.312794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a75b0-6fc0-4512-bd9c-7e729d0d69aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:20.317087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:20.329690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5c5a75b0-6fc0-4512-bd9c-7e729d0d69aa None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:20.330992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5c5a75b0-6fc0-4512-bd9c-7e729d0d69aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:20.411228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 625/1248] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:20.434710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ec03f-47d7-4293-8d7c-2bf3786ef1fc None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:20.454682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:20.549191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6ec03f-47d7-4293-8d7c-2bf3786ef1fc None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:20.585081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540647f-1f37-4b97-9408-d23a1863f44d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:20.585807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540647f-1f37-4b97-9408-d23a1863f44d None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:20.586082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540647f-1f37-4b97-9408-d23a1863f44d None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:20.586351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540647f-1f37-4b97-9408-d23a1863f44d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:20.598308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540647f-1f37-4b97-9408-d23a1863f44d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-a540647f-1f37-4b97-9408-d23a1863f44d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:03:20.606036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:20.607154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:20.608184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 624/1249] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:03:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:20.614020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a75b0-6fc0-4512-bd9c-7e729d0d69aa None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:20.719379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a75b0-6fc0-4512-bd9c-7e729d0d69aa None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:20.752852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:20.760647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:20.761444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:20.761734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:20.762006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:20.762759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:20.763066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:20.764459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:20.773720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 626/1250] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:03:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:20.780360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540647f-1f37-4b97-9408-d23a1863f44d None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:20.863865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:20.888191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540647f-1f37-4b97-9408-d23a1863f44d None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:20.923570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:20.924428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:20.924753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:20.925084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:20.936081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:20.937577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:20.956543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:20.957789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:20.958850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:20.958850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:20.958850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:20.958850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:20.958850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 625/1251] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:20.964162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:21.093208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d028e336-844d-4d3c-b30c-75e2fa3a3fe5 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:21.155113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:21.155921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:21.156314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:21.156688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:21.157543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:21.157866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:21.159323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:21.173362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 627/1252] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:21.179923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:21.294441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e6e518-edbd-4d67-b5d0-9e1e8fc0e106 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:21.322396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:21.330878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:21.331730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:21.332011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:21.332286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:21.339705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:21.341251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:03:21.356742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:21.357997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:21.358949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 626/1253] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:21.364389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:21.430736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:21.482558 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2944e-8fa3-43b2-8049-930135767f87 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:21.523701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:21.524372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:21.524659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:21.524928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:21.525900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:21.526082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:21.527531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:21.546598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 628/1254] 10.222.0.69 () {56 vars in 1143 bytes} [Fri May 20 09:03:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:21.552974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:21.670262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58a5073-9773-4007-8fd7-f7c5334e0613 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:21.714657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295d3e8-e737-415f-9b3e-be81cbcb50e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:21.715419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295d3e8-e737-415f-9b3e-be81cbcb50e6 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:21.715799 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295d3e8-e737-415f-9b3e-be81cbcb50e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:21.716081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f295d3e8-e737-415f-9b3e-be81cbcb50e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:21.786340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:21.799717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295d3e8-e737-415f-9b3e-be81cbcb50e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:21.801490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f295d3e8-e737-415f-9b3e-be81cbcb50e6 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:21.905516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 627/1255] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:03:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:03:21.912295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:22.019091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a18fe-12eb-48ae-9ddf-e0345602bf69 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:22.033451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:22.056592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:22.057518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:22.057789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:22.058060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:22.069118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:22.070693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:22.088595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:22.089726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:22.091118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:22.091118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:22.091118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:22.091118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:22.091118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 629/1256] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:22.096169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295d3e8-e737-415f-9b3e-be81cbcb50e6 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:22.200275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:22.216186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f295d3e8-e737-415f-9b3e-be81cbcb50e6 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:22.255158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:22.255862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:22.256145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:22.256421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:22.257157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:22.257465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:22.258887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:22.271280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 628/1257] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:22.277775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:22.377692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31454ded-5dba-4abc-a681-fe7bf958860b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:22.413539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:22.414391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:22.414672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:22.414948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:22.423511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:22.425153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:03:22.440050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:22.441048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:22.441970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:03:22.441970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:03:22.441970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:03:22.441970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:03:22.441970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:03:22.441970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:03:22.441970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:22.441970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:22.444532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 630/1258] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:03:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:22.450838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:22.570420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e85d3b-f149-4603-8079-247257aba2e9 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:22.607817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d532b47e-680e-4e18-89e9-ce17c2fe3a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:22.608484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532b47e-680e-4e18-89e9-ce17c2fe3a5e None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:22.608774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532b47e-680e-4e18-89e9-ce17c2fe3a5e None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:22.609047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d532b47e-680e-4e18-89e9-ce17c2fe3a5e None admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:22.609850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532b47e-680e-4e18-89e9-ce17c2fe3a5e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:22.610169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532b47e-680e-4e18-89e9-ce17c2fe3a5e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:22.611624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d532b47e-680e-4e18-89e9-ce17c2fe3a5e None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:22.624236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 629/1259] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:03:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:22.630510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:22.661421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:22.741731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f8a87-7db5-4ad2-ad5d-a7ede670f4ba None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:22.782798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0be64e-a860-495e-aa6e-9ea9c0377da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:22.783552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0be64e-a860-495e-aa6e-9ea9c0377da0 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:22.783836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0be64e-a860-495e-aa6e-9ea9c0377da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:22.784108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0be64e-a860-495e-aa6e-9ea9c0377da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:22.784963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0be64e-a860-495e-aa6e-9ea9c0377da0 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=569782752acb4085ad742330af999bc2, user.id=6def7fd117dc4b7598a1864e63f09427) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:22.785265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0be64e-a860-495e-aa6e-9ea9c0377da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:22.787377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0be64e-a860-495e-aa6e-9ea9c0377da0 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:22.793295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 631/1260] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:03:22 2022] GET /identity/v3/role_assignments?user.id=6def7fd117dc4b7598a1864e63f09427&scope.project.id=569782752acb4085ad742330af999bc2&role.id=6dca675ade814980a252f800fb4fa3ad => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:24.988191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded83b7-250d-4d89-b619-489437e45bab None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:24.988790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded83b7-250d-4d89-b619-489437e45bab None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:24.988790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded83b7-250d-4d89-b619-489437e45bab None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:24.990119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 630/1261] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:24.996623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b27644-ce79-4c5f-adbb-c978c6b518cd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:24.997347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b27644-ce79-4c5f-adbb-c978c6b518cd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:24.997497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b27644-ce79-4c5f-adbb-c978c6b518cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:25.025761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d1b27644-ce79-4c5f-adbb-c978c6b518cd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:25.026875 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d1b27644-ce79-4c5f-adbb-c978c6b518cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:25.096190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 632/1262] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:25.121242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded83b7-250d-4d89-b619-489437e45bab None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:25.132442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:25.230653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded83b7-250d-4d89-b619-489437e45bab None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:25.265704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88abdac-5a12-4eaf-9722-176daf60e174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:25.266480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88abdac-5a12-4eaf-9722-176daf60e174 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:25.266758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88abdac-5a12-4eaf-9722-176daf60e174 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:25.267025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88abdac-5a12-4eaf-9722-176daf60e174 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:25.277832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88abdac-5a12-4eaf-9722-176daf60e174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-c88abdac-5a12-4eaf-9722-176daf60e174 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:03:25.285228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:25.286677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:25.287657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 631/1263] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:03:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:25.293697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b27644-ce79-4c5f-adbb-c978c6b518cd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:25.312028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:25.392319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b27644-ce79-4c5f-adbb-c978c6b518cd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:25.427631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:25.428277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:25.428554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:25.428820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:25.429562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:25.429864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:25.431233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:25.439335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 633/1264] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:03:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:25.445919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88abdac-5a12-4eaf-9722-176daf60e174 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:25.527456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:25.569890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88abdac-5a12-4eaf-9722-176daf60e174 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:25.608362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:25.609164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:25.609447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:25.609727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:25.620680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:25.622154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:25.639463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:25.640434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:25.641656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:25.641656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:25.641656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:25.641656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:25.641656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 632/1265] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:25.646726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:25.726634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:25.752235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8fbd0-c01c-41e4-80ae-bd86dd25db9f None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:25.790192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:25.790909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:25.791189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:25.791488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:25.792209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:25.792568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:25.793936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:25.806275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 634/1266] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:25.814501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:25.933935 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230ef2a-2b88-4e04-a78f-d2045ca975d5 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:25.959838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:25.975529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:25.976330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:25.976616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:25.976894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] PATH_INFO: `/v3/users/placement` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:25.984576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:25.986055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:03:26.000324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:26.001739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:26.002684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 633/1267] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:26.007710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:26.115787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0071d66-a7fa-4a1a-9b1d-c596cd3c5559 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:26.150643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:26.151308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:26.151609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:26.151885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:26.152726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:26.153038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:26.154468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:26.163459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:26.177312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 635/1268] 10.222.0.69 () {56 vars in 1143 bytes} [Fri May 20 09:03:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:26.183169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:26.306212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c2f72-8b48-414d-9301-5f6f6469c9ee None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:26.341582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:26.342380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:26.342660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:26.342928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:26.354106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:26.355601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:26.372694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:26.373682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:26.375152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:26.375152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:26.375152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:26.375152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:26.375152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 634/1269] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:26.382366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:26.487923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7e6ade-aadf-48f5-9259-9aa449405d25 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:26.523821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:26.524585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:26.524789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:26.525051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:26.525796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:26.526105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:26.527511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:26.539239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 636/1270] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:26.545150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:26.587379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:26.663059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72857424-f979-4bc5-ba57-5818ee60afae None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:26.697730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:26.698505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:26.698784 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:26.699054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:26.705985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:26.707462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:03:26.721494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:26.722502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:26.723504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 635/1271] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:03:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:26.728798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:26.786590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:26.829879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828acb3a-64b2-43a9-adb9-c8a11249bcbc None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:26.864426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87bb59-33d7-45ae-8b9e-fc89dad89d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:26.864998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87bb59-33d7-45ae-8b9e-fc89dad89d86 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:26.865275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87bb59-33d7-45ae-8b9e-fc89dad89d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:26.865544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87bb59-33d7-45ae-8b9e-fc89dad89d86 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:26.866325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87bb59-33d7-45ae-8b9e-fc89dad89d86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:26.866626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87bb59-33d7-45ae-8b9e-fc89dad89d86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:26.868078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87bb59-33d7-45ae-8b9e-fc89dad89d86 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:26.880914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 637/1272] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:03:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:26.887344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:26.978689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:26.988284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3798ee88-d800-4d6c-b9b8-edd0ec950596 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:27.022374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9340832d-4389-4433-b12b-c4de62cea138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:27.023301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340832d-4389-4433-b12b-c4de62cea138 None admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:27.023563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340832d-4389-4433-b12b-c4de62cea138 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:27.023829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9340832d-4389-4433-b12b-c4de62cea138 None admin] PATH_INFO: `/v3/projects/569782752acb4085ad742330af999bc2/users/6def7fd117dc4b7598a1864e63f09427/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:27.026044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340832d-4389-4433-b12b-c4de62cea138 None admin] RBAC: Authorizing `identity:create_grant(project_id=569782752acb4085ad742330af999bc2, user_id=6def7fd117dc4b7598a1864e63f09427, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:27.027500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9340832d-4389-4433-b12b-c4de62cea138 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:27.046624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 636/1273] 10.222.0.69 () {58 vars in 1439 bytes} [Fri May 20 09:03:26 2022] PUT /identity/v3/projects/569782752acb4085ad742330af999bc2/users/6def7fd117dc4b7598a1864e63f09427/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:03:29.201850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e1a05-f6dc-4285-866a-19a60a407368 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:29.202402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e1a05-f6dc-4285-866a-19a60a407368 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:29.202402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3e1a05-f6dc-4285-866a-19a60a407368 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:29.203707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 638/1274] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:29.210006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92cc3-e8cf-4ad3-8408-b9efd0dfb0b0 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:29.210434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92cc3-e8cf-4ad3-8408-b9efd0dfb0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:29.210610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e92cc3-e8cf-4ad3-8408-b9efd0dfb0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:29.230056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-93e92cc3-e8cf-4ad3-8408-b9efd0dfb0b0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:29.231290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-93e92cc3-e8cf-4ad3-8408-b9efd0dfb0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:29.302038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 637/1275] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:29.326723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e1a05-f6dc-4285-866a-19a60a407368 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:29.436975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3e1a05-f6dc-4285-866a-19a60a407368 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:29.454285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:29.473008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106beec-b5d7-4a0b-8353-81c5b324c512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:29.473786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106beec-b5d7-4a0b-8353-81c5b324c512 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:29.474068 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106beec-b5d7-4a0b-8353-81c5b324c512 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:29.474333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0106beec-b5d7-4a0b-8353-81c5b324c512 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:29.484812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0106beec-b5d7-4a0b-8353-81c5b324c512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-0106beec-b5d7-4a0b-8353-81c5b324c512 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(role_id) May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 20 09:03:29.492221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:29.493253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:29.494446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 639/1276] 10.222.0.69 () {56 vars in 1096 bytes} [Fri May 20 09:03:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:29.499712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e92cc3-e8cf-4ad3-8408-b9efd0dfb0b0 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:29.586997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:29.606109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e92cc3-e8cf-4ad3-8408-b9efd0dfb0b0 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:29.640431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:29.641079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:29.641363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:29.641631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:29.642380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:29.642681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:29.644086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:29.652104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 638/1277] 10.222.0.69 () {56 vars in 1099 bytes} [Fri May 20 09:03:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:29.660887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106beec-b5d7-4a0b-8353-81c5b324c512 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:29.764945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0106beec-b5d7-4a0b-8353-81c5b324c512 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:29.801301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:29.802081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:29.802370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:29.802621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:29.813274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:29.814706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:29.831652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:29.832688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:29.833939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:29.833939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:29.833939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:29.833939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:29.833939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 640/1278] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:03:29.838652 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:29.953169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153cc6cc-8361-4b7c-b155-23ea90c7f142 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:29.986657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:29.987287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:29.987611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:29.987881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:29.988600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:29.988903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:29.990273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:30.001844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 639/1279] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:30.007419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:30.039009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:30.124846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfa1d8-131b-42ca-b656-b5f07297e73e None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:30.166840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:30.167703 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:30.167990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:30.168267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:30.176064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:30.177531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_user(user_id) May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 20 09:03:30.191845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:30.192934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:30.193942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 641/1280] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:30.199788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:30.204181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:30.303384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd02e5d-7f23-4bfa-a2da-aa4652c0bcfb None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:30.339421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:30.340085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:30.340365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:30.340639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:30.341485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:30.341795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:30.343234 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:30.360251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 640/1281] 10.222.0.69 () {56 vars in 1143 bytes} [Fri May 20 09:03:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:30.369095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:30.468887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:30.480066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61bf560-f8c7-4c9e-b462-77493eef049b None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:30.516573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d768d-fc2a-413f-b7ad-15a6c86c2194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:30.517172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d768d-fc2a-413f-b7ad-15a6c86c2194 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:30.517450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d768d-fc2a-413f-b7ad-15a6c86c2194 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:30.517748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6d768d-fc2a-413f-b7ad-15a6c86c2194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:30.588588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d768d-fc2a-413f-b7ad-15a6c86c2194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:30.589992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6d768d-fc2a-413f-b7ad-15a6c86c2194 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:30.687596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:30.702870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 642/1282] 10.222.0.69 () {58 vars in 1282 bytes} [Fri May 20 09:03:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 20 09:03:30.711626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:30.777070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:30.817094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518175f-4876-4675-a47c-17fda6543215 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:30.852057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:30.852808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:30.853113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:30.853381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:30.863610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:30.865061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 20 09:03:30.882181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:30.883187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:30.884331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 20 09:03:30.884331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 20 09:03:30.884331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 20 09:03:30.884331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:30.884331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 641/1283] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:03:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 20 09:03:30.889237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d768d-fc2a-413f-b7ad-15a6c86c2194 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:30.992024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6d768d-fc2a-413f-b7ad-15a6c86c2194 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:31.028802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38daf85-512d-4663-8020-985885826b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:31.029442 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38daf85-512d-4663-8020-985885826b12 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:31.029717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38daf85-512d-4663-8020-985885826b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:31.029986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38daf85-512d-4663-8020-985885826b12 None admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:31.030720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38daf85-512d-4663-8020-985885826b12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:31.031023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38daf85-512d-4663-8020-985885826b12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:31.032418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38daf85-512d-4663-8020-985885826b12 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:31.043948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 643/1284] 10.222.0.69 () {56 vars in 1109 bytes} [Fri May 20 09:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:31.052836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:31.165805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c7aed4-b3e8-4697-8812-6c2912474f34 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:31.179282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:31.202814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:31.203641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:31.203921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:31.204198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:31.211010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:31.212534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(project_id) May 20 09:03:31.226377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application with Lock( May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:31.227411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: ERROR keystone.server.flask.application May 20 09:03:31.228579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 642/1285] 10.222.0.69 () {56 vars in 1111 bytes} [Fri May 20 09:03:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:31.235974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38daf85-512d-4663-8020-985885826b12 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:31.303352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:31.356723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38daf85-512d-4663-8020-985885826b12 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:31.394474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b270596b-fe9c-41fd-8466-0409f47d2464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:31.395136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270596b-fe9c-41fd-8466-0409f47d2464 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:31.395441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270596b-fe9c-41fd-8466-0409f47d2464 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:31.395715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b270596b-fe9c-41fd-8466-0409f47d2464 None admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:31.396510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270596b-fe9c-41fd-8466-0409f47d2464 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:31.396816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270596b-fe9c-41fd-8466-0409f47d2464 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:31.398251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b270596b-fe9c-41fd-8466-0409f47d2464 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:31.410941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 644/1286] 10.222.0.69 () {56 vars in 1148 bytes} [Fri May 20 09:03:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:31.417300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:31.526724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406102bb-69e5-43aa-bea9-dd8e319b4b25 None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:31.561576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441faba3-4589-4b33-8f41-7490bc7ab3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:31.562226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441faba3-4589-4b33-8f41-7490bc7ab3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:31.562510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441faba3-4589-4b33-8f41-7490bc7ab3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:31.562782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441faba3-4589-4b33-8f41-7490bc7ab3c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:31.563640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441faba3-4589-4b33-8f41-7490bc7ab3c1 None admin] RBAC: Adding query filter params (role.id=6dca675ade814980a252f800fb4fa3ad, scope.project.id=569782752acb4085ad742330af999bc2, user.id=6def7fd117dc4b7598a1864e63f09427) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:31.563958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441faba3-4589-4b33-8f41-7490bc7ab3c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:31.565426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441faba3-4589-4b33-8f41-7490bc7ab3c1 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:31.571380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 643/1287] 10.222.0.69 () {56 vars in 1374 bytes} [Fri May 20 09:03:31 2022] GET /identity/v3/role_assignments?user.id=6def7fd117dc4b7598a1864e63f09427&scope.project.id=569782752acb4085ad742330af999bc2&role.id=6dca675ade814980a252f800fb4fa3ad => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:33.762243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221c26-89b8-432f-96d2-8e440ec6c0b2 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:33.762809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221c26-89b8-432f-96d2-8e440ec6c0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:33.762809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac221c26-89b8-432f-96d2-8e440ec6c0b2 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:33.764195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 645/1288] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:33.770216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1135c-6a58-48c1-8ca0-468e5a870197 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:33.770678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1135c-6a58-48c1-8ca0-468e5a870197 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:33.770875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d1135c-6a58-48c1-8ca0-468e5a870197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:33.789939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e2d1135c-6a58-48c1-8ca0-468e5a870197 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:33.791126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e2d1135c-6a58-48c1-8ca0-468e5a870197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:33.829142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:33.882144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 644/1289] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:33.911395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac221c26-89b8-432f-96d2-8e440ec6c0b2 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:34.026918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac221c26-89b8-432f-96d2-8e440ec6c0b2 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:34.038030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:34.065288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:34.066064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:34.066344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:34.066609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] PATH_INFO: `/v3/services/placement` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:34.067248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:34.068621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:34.081899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:34.083058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:34.084165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 646/1290] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:03:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:34.089276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1135c-6a58-48c1-8ca0-468e5a870197 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:34.197341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d1135c-6a58-48c1-8ca0-468e5a870197 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:34.250184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a9fba-0ad2-4449-85e1-f1f7466aeeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:34.250853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9fba-0ad2-4449-85e1-f1f7466aeeed None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:34.251139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9fba-0ad2-4449-85e1-f1f7466aeeed None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:34.251438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a9fba-0ad2-4449-85e1-f1f7466aeeed None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:34.252158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a9fba-0ad2-4449-85e1-f1f7466aeeed None admin] RBAC: Adding query filter params (name=placement) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:34.252463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a9fba-0ad2-4449-85e1-f1f7466aeeed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:34.253859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a9fba-0ad2-4449-85e1-f1f7466aeeed None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:34.259605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 645/1291] 10.222.0.69 () {56 vars in 1116 bytes} [Fri May 20 09:03:34 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:34.267988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:34.371387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7ba48-5d9e-4ad1-b64b-b2e7bffbb2df None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:34.405394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f8b790-103f-4c30-9c43-7eebbf18602a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:34.406030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8b790-103f-4c30-9c43-7eebbf18602a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:34.406310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8b790-103f-4c30-9c43-7eebbf18602a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:34.406587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f8b790-103f-4c30-9c43-7eebbf18602a None admin] PATH_INFO: `/v3/services` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:34.407298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8b790-103f-4c30-9c43-7eebbf18602a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:34.407621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8b790-103f-4c30-9c43-7eebbf18602a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:34.408992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f8b790-103f-4c30-9c43-7eebbf18602a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:34.413896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 647/1292] 10.222.0.69 () {56 vars in 1116 bytes} [Fri May 20 09:03:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:36.669196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a74229-924b-42e2-a53b-bc50f9d4cbf5 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:36.669752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a74229-924b-42e2-a53b-bc50f9d4cbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:36.669752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a74229-924b-42e2-a53b-bc50f9d4cbf5 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:36.671144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 646/1293] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:36.676610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfa5c8-14a3-4168-a82c-a0fb0764ab0a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:36.677034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfa5c8-14a3-4168-a82c-a0fb0764ab0a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:36.677146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfa5c8-14a3-4168-a82c-a0fb0764ab0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:36.694552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-04cfa5c8-14a3-4168-a82c-a0fb0764ab0a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:36.695752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-04cfa5c8-14a3-4168-a82c-a0fb0764ab0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:36.726209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:36.771770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 648/1294] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:36.794598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a74229-924b-42e2-a53b-bc50f9d4cbf5 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:36.829524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:36.906931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a74229-924b-42e2-a53b-bc50f9d4cbf5 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:36.941894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0a2867-a50f-4056-b7af-2d6a056f2de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:36.942483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2867-a50f-4056-b7af-2d6a056f2de5 None admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:36.942764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2867-a50f-4056-b7af-2d6a056f2de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:36.943041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0a2867-a50f-4056-b7af-2d6a056f2de5 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:36.943798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a2867-a50f-4056-b7af-2d6a056f2de5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:36.945177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0a2867-a50f-4056-b7af-2d6a056f2de5 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:36.967310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 647/1295] 10.222.0.69 () {60 vars in 1141 bytes} [Fri May 20 09:03:36 2022] POST /identity/v3/services => generated 246 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:03:39.172485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c397b-629c-4568-a64c-fc8f467a4b05 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:39.173362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c397b-629c-4568-a64c-fc8f467a4b05 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:39.173362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72c397b-629c-4568-a64c-fc8f467a4b05 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:39.175577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 649/1296] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:39.181864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eef33-a3b2-414c-9a3e-4e31f8baa55d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:39.182277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eef33-a3b2-414c-9a3e-4e31f8baa55d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:39.182389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422eef33-a3b2-414c-9a3e-4e31f8baa55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:39.200356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-422eef33-a3b2-414c-9a3e-4e31f8baa55d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:39.201544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-422eef33-a3b2-414c-9a3e-4e31f8baa55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:39.276550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 648/1297] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:39 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 20 09:03:39.299969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c397b-629c-4568-a64c-fc8f467a4b05 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:39.415597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72c397b-629c-4568-a64c-fc8f467a4b05 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:39.450379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:39.451099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:39.451409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:39.451687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:39.452335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:39.453708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:39.465735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:39.466870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:39.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 650/1298] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:03:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:39.473804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eef33-a3b2-414c-9a3e-4e31f8baa55d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:39.494371 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:39.583917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422eef33-a3b2-414c-9a3e-4e31f8baa55d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:39.621251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:39.621912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:39.622206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:39.622469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:39.623187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:39.623537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:39.624940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:39.630190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 649/1299] 10.222.0.69 () {56 vars in 1116 bytes} [Fri May 20 09:03:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:39.638041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:39.672824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:39.743371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4a4e8-5225-4447-83c6-bcf6316d0d1a None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:39.778820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57ef8c-9862-47a0-92a8-8e728e553d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:39.779565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57ef8c-9862-47a0-92a8-8e728e553d79 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:39.779849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57ef8c-9862-47a0-92a8-8e728e553d79 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:39.780123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57ef8c-9862-47a0-92a8-8e728e553d79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:39.780925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57ef8c-9862-47a0-92a8-8e728e553d79 None admin] RBAC: Adding query filter params (interface=public, service_id=2ff59fccf02f42ea80e8d18ccd0f850f, region_id=RegionOne) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:39.781231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57ef8c-9862-47a0-92a8-8e728e553d79 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:39.782593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57ef8c-9862-47a0-92a8-8e728e553d79 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:39.788163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 651/1300] 10.222.0.69 () {56 vars in 1251 bytes} [Fri May 20 09:03:39 2022] GET /identity/v3/endpoints?service_id=2ff59fccf02f42ea80e8d18ccd0f850f&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:39.794021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:39.879680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:39.900326 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269050e-3a86-4b34-b46c-a90c878d884b None admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:39.936164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eafd0b-93ce-435f-8da3-7fe468372fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:39.936812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eafd0b-93ce-435f-8da3-7fe468372fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:39.937100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eafd0b-93ce-435f-8da3-7fe468372fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:39.937374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eafd0b-93ce-435f-8da3-7fe468372fd7 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:39.938056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eafd0b-93ce-435f-8da3-7fe468372fd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:39.939477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eafd0b-93ce-435f-8da3-7fe468372fd7 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:39.946811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 650/1301] 10.222.0.69 () {56 vars in 1087 bytes} [Fri May 20 09:03:39 2022] GET /identity/v3/services => generated 2191 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:03:42.121215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083aaca-3587-4503-8c5a-9a0546bfac81 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:42.121764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083aaca-3587-4503-8c5a-9a0546bfac81 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:42.121764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3083aaca-3587-4503-8c5a-9a0546bfac81 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:42.123041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 652/1302] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:03:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:03:42.129469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293b32-472c-4c4f-8826-a5d5186234ca None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:42.130176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293b32-472c-4c4f-8826-a5d5186234ca None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:42.130339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293b32-472c-4c4f-8826-a5d5186234ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:42.155735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-61293b32-472c-4c4f-8826-a5d5186234ca None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:03:42.156928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-61293b32-472c-4c4f-8826-a5d5186234ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:03:42.226860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 651/1303] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:03:42 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 20 09:03:42.256208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083aaca-3587-4503-8c5a-9a0546bfac81 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:42.361621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:42.369256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3083aaca-3587-4503-8c5a-9a0546bfac81 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:42.404912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:42.405645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:42.405922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:42.406199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:42.406860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:42.408268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application Traceback (most recent call last): May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 20 09:03:42.420537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._get_service(service_id) May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 20 09:03:42.421685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.get_or_create( May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application with Lock( May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self._enter() May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application return self.creator() May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application created_value = creator( May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: ERROR keystone.server.flask.application May 20 09:03:42.423326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 653/1304] 10.222.0.69 () {56 vars in 1117 bytes} [Fri May 20 09:03:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 20 09:03:42.431199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293b32-472c-4c4f-8826-a5d5186234ca None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:42.510077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:42.545246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293b32-472c-4c4f-8826-a5d5186234ca None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:42.581848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9c3a64-3fab-42c8-a620-a991fbb73d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:42.582508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c3a64-3fab-42c8-a620-a991fbb73d02 None admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:42.582779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c3a64-3fab-42c8-a620-a991fbb73d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:42.583063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9c3a64-3fab-42c8-a620-a991fbb73d02 None admin] PATH_INFO: `/v3/services` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:42.583809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c3a64-3fab-42c8-a620-a991fbb73d02 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:03:42.584094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c3a64-3fab-42c8-a620-a991fbb73d02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:42.585508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9c3a64-3fab-42c8-a620-a991fbb73d02 None admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:42.590958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 652/1305] 10.222.0.69 () {56 vars in 1116 bytes} [Fri May 20 09:03:42 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:03:42.598930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:03:42.708437 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc4da7-aa2d-4052-99ca-2c57174d91d7 None admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:03:42.743163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589df8e-e742-4317-83a3-3c7c1d742c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:03:42.743878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589df8e-e742-4317-83a3-3c7c1d742c10 None admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:03:42.744162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589df8e-e742-4317-83a3-3c7c1d742c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:03:42.744434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589df8e-e742-4317-83a3-3c7c1d742c10 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:03:42.745176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589df8e-e742-4317-83a3-3c7c1d742c10 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:03:42.746560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589df8e-e742-4317-83a3-3c7c1d742c10 None admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:03:42.769572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 654/1306] 10.222.0.69 () {60 vars in 1144 bytes} [Fri May 20 09:03:42 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:03:52.588112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=30 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:03:52.765585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:04:46.399089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bc67f-7d16-4493-8906-7d4ec374bda7 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:46.399781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bc67f-7d16-4493-8906-7d4ec374bda7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:46.399781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bc67f-7d16-4493-8906-7d4ec374bda7 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:46.406698 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8ba98-46b3-4766-93fa-da04388cf132 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:46.407164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8ba98-46b3-4766-93fa-da04388cf132 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:46.407298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8ba98-46b3-4766-93fa-da04388cf132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:46.409851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 653/1307] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:04:46 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:04:46.565099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-26d8ba98-46b3-4766-93fa-da04388cf132 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:04:46.566326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-26d8ba98-46b3-4766-93fa-da04388cf132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:46.827582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 655/1308] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:04:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:04:46.884324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3723d8-afcc-493b-8c88-7e2413239106 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:46.884616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3723d8-afcc-493b-8c88-7e2413239106 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:46.884894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3723d8-afcc-493b-8c88-7e2413239106 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:46.886436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 654/1309] 10.222.0.69 () {54 vars in 964 bytes} [Fri May 20 09:04:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:04:46.896227 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339196de-f120-4e94-ba52-199b599e6a53 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:46.896522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339196de-f120-4e94-ba52-199b599e6a53 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:46.896858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339196de-f120-4e94-ba52-199b599e6a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:47.051002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-339196de-f120-4e94-ba52-199b599e6a53 None None] MFA Rules not processed for user `349e99511c7c4307b27dc90b01f2dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:04:47.052677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-339196de-f120-4e94-ba52-199b599e6a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:47.084515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:04:47.153146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 656/1310] 10.222.0.69 () {58 vars in 1061 bytes} [Fri May 20 09:04:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:04:47.162128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3723d8-afcc-493b-8c88-7e2413239106 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:04:47.164257 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7d3723d8-afcc-493b-8c88-7e2413239106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:47.277052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3723d8-afcc-493b-8c88-7e2413239106 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:04:47.358104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8240a8-5176-4e5b-8fff-39d996311a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:04:47.359046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8240a8-5176-4e5b-8fff-39d996311a95 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:47.359655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8240a8-5176-4e5b-8fff-39d996311a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:47.360111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8240a8-5176-4e5b-8fff-39d996311a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:47.362214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8d8240a8-5176-4e5b-8fff-39d996311a95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:47.423253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:04:47.482439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8240a8-5176-4e5b-8fff-39d996311a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:04:47.484364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8240a8-5176-4e5b-8fff-39d996311a95 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:04:47.504687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 655/1311] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:04:50.027318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a923f2-8c6c-4572-a33f-29b5173f2e21 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:50.027828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a923f2-8c6c-4572-a33f-29b5173f2e21 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:50.027919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a923f2-8c6c-4572-a33f-29b5173f2e21 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:50.029250 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 657/1312] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:04:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:04:50.035281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23039fd5-a005-4501-bbaa-3d85c0b7a495 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:50.035774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23039fd5-a005-4501-bbaa-3d85c0b7a495 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:50.035854 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23039fd5-a005-4501-bbaa-3d85c0b7a495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:50.112550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-23039fd5-a005-4501-bbaa-3d85c0b7a495 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:04:50.113774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-23039fd5-a005-4501-bbaa-3d85c0b7a495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:50.216006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 656/1313] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:04:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:04:50.253693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a923f2-8c6c-4572-a33f-29b5173f2e21 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:04:50.255375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e9a923f2-8c6c-4572-a33f-29b5173f2e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:50.360777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a923f2-8c6c-4572-a33f-29b5173f2e21 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:04:50.434307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:04:50.436982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c6fa45-bec6-41df-9e47-80d52191ae73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:04:50.437751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6fa45-bec6-41df-9e47-80d52191ae73 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:50.438046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6fa45-bec6-41df-9e47-80d52191ae73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:50.438327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c6fa45-bec6-41df-9e47-80d52191ae73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:50.440637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-48c6fa45-bec6-41df-9e47-80d52191ae73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:50.552993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6fa45-bec6-41df-9e47-80d52191ae73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:04:50.554441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c6fa45-bec6-41df-9e47-80d52191ae73 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:04:50.574726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 658/1314] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:04:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:04:53.094862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9cb7-db72-4c49-84a4-cd5d00d493e6 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:53.095462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9cb7-db72-4c49-84a4-cd5d00d493e6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:53.095462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881a9cb7-db72-4c49-84a4-cd5d00d493e6 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:53.096844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 657/1315] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:04:53.103126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949d73-685d-48c2-a1b1-13368331b103 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:53.103938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949d73-685d-48c2-a1b1-13368331b103 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:53.104035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e949d73-685d-48c2-a1b1-13368331b103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:53.171133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6e949d73-685d-48c2-a1b1-13368331b103 None None] MFA Rules not processed for user `bdd8156d03f1489f99984f234d7d8ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:04:53.172363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6e949d73-685d-48c2-a1b1-13368331b103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:53.224625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:04:53.276557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 659/1316] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:04:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:04:53.320065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a9cb7-db72-4c49-84a4-cd5d00d493e6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:04:53.321604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-881a9cb7-db72-4c49-84a4-cd5d00d493e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:53.331699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:04:53.427595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881a9cb7-db72-4c49-84a4-cd5d00d493e6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:04:53.507694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c9754-41d4-4c60-ae96-0d457a441acb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:04:53.508335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c9754-41d4-4c60-ae96-0d457a441acb service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:53.508613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c9754-41d4-4c60-ae96-0d457a441acb service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:53.508880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c9754-41d4-4c60-ae96-0d457a441acb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:53.510605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-610c9754-41d4-4c60-ae96-0d457a441acb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:53.573057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:04:53.643532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c9754-41d4-4c60-ae96-0d457a441acb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:04:53.644990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c9754-41d4-4c60-ae96-0d457a441acb service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:04:53.666588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 658/1317] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:04:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:04:56.404276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68c1b1-a47a-42fe-ac4f-bd2023ebee62 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:56.404865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68c1b1-a47a-42fe-ac4f-bd2023ebee62 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:56.404865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68c1b1-a47a-42fe-ac4f-bd2023ebee62 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:56.406224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 660/1318] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:04:56.412958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f253d59-98f6-4f71-b1bb-54a3402b05b5 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:56.413668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f253d59-98f6-4f71-b1bb-54a3402b05b5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:56.413809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f253d59-98f6-4f71-b1bb-54a3402b05b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:56.484664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0f253d59-98f6-4f71-b1bb-54a3402b05b5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:04:56.485856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0f253d59-98f6-4f71-b1bb-54a3402b05b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:56.580896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 659/1319] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:04:56 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:04:56.618762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68c1b1-a47a-42fe-ac4f-bd2023ebee62 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:04:56.620362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ea68c1b1-a47a-42fe-ac4f-bd2023ebee62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:56.725346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68c1b1-a47a-42fe-ac4f-bd2023ebee62 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:04:56.800656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787d2d8-19dd-42bf-a6bb-a91153a45cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:04:56.801294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787d2d8-19dd-42bf-a6bb-a91153a45cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:56.801569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787d2d8-19dd-42bf-a6bb-a91153a45cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:56.801840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787d2d8-19dd-42bf-a6bb-a91153a45cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:56.803504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7787d2d8-19dd-42bf-a6bb-a91153a45cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:56.811167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:04:56.921796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787d2d8-19dd-42bf-a6bb-a91153a45cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:04:56.924212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787d2d8-19dd-42bf-a6bb-a91153a45cf5 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:04:56.944722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 661/1320] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:04:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:04:59.828553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e563e-fc9f-4dff-aa19-1036ed562596 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:59.829116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e563e-fc9f-4dff-aa19-1036ed562596 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:59.829116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e563e-fc9f-4dff-aa19-1036ed562596 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:59.830484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 660/1321] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:04:59.836007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cf063-d25b-4db7-a13f-1d098380a575 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:04:59.836424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cf063-d25b-4db7-a13f-1d098380a575 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:04:59.836576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1cf063-d25b-4db7-a13f-1d098380a575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:04:59.901338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1f1cf063-d25b-4db7-a13f-1d098380a575 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:04:59.902566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1f1cf063-d25b-4db7-a13f-1d098380a575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:04:59.961075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:00.000103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 662/1322] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:04:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:00.035870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e563e-fc9f-4dff-aa19-1036ed562596 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:00.037405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2b0e563e-fc9f-4dff-aa19-1036ed562596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:00.057465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:00.144220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e563e-fc9f-4dff-aa19-1036ed562596 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:00.221306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea22ba-c1df-4064-8dc7-248b25153fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:00.221970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea22ba-c1df-4064-8dc7-248b25153fff service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:00.222248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea22ba-c1df-4064-8dc7-248b25153fff service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:00.222491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea22ba-c1df-4064-8dc7-248b25153fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:00.224313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3aea22ba-c1df-4064-8dc7-248b25153fff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:00.298195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:00.340189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea22ba-c1df-4064-8dc7-248b25153fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:00.341951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea22ba-c1df-4064-8dc7-248b25153fff service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:00.363521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 661/1323] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:02.603112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b32c6-4ecc-46f3-8ed3-a10229af363a None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:02.603835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b32c6-4ecc-46f3-8ed3-a10229af363a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:02.603835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787b32c6-4ecc-46f3-8ed3-a10229af363a None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:02.605135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 663/1324] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:02.611288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662bd2-4663-4800-ad3b-6efa13792ae2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:02.611843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662bd2-4663-4800-ad3b-6efa13792ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:02.611843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81662bd2-4663-4800-ad3b-6efa13792ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:02.677358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-81662bd2-4663-4800-ad3b-6efa13792ae2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:02.678749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-81662bd2-4663-4800-ad3b-6efa13792ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:02.778434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 662/1325] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:02.814328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b32c6-4ecc-46f3-8ed3-a10229af363a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:02.815950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-787b32c6-4ecc-46f3-8ed3-a10229af363a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:02.923010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787b32c6-4ecc-46f3-8ed3-a10229af363a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:02.999356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e9de12-8196-47bd-9ebb-752fe12b669a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:03.000012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9de12-8196-47bd-9ebb-752fe12b669a service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:03.000296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9de12-8196-47bd-9ebb-752fe12b669a service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:03.000587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e9de12-8196-47bd-9ebb-752fe12b669a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:03.002219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f4e9de12-8196-47bd-9ebb-752fe12b669a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:03.018688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:03.118331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9de12-8196-47bd-9ebb-752fe12b669a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:03.119904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e9de12-8196-47bd-9ebb-752fe12b669a service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:03.137968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 664/1326] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:05.379131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17681596-9288-48ce-9efc-698b6a1627b3 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:05.379724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17681596-9288-48ce-9efc-698b6a1627b3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:05.379724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17681596-9288-48ce-9efc-698b6a1627b3 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:05.381129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 663/1327] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:05.386612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343f560-77cf-4fe5-8dc2-cf29f5611283 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:05.387053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343f560-77cf-4fe5-8dc2-cf29f5611283 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:05.387148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3343f560-77cf-4fe5-8dc2-cf29f5611283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:05.452131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3343f560-77cf-4fe5-8dc2-cf29f5611283 None None] MFA Rules not processed for user `bdd8156d03f1489f99984f234d7d8ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:05.453375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3343f560-77cf-4fe5-8dc2-cf29f5611283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:05.523225 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:05.556301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 665/1328] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:05 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:05.593258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17681596-9288-48ce-9efc-698b6a1627b3 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:05.594826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-17681596-9288-48ce-9efc-698b6a1627b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:05.626785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:05.706403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17681596-9288-48ce-9efc-698b6a1627b3 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:05.788156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fff24b-5f3d-4d94-bce8-03706b5fcb0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:05.788848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fff24b-5f3d-4d94-bce8-03706b5fcb0b service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:05.789105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fff24b-5f3d-4d94-bce8-03706b5fcb0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:05.789377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fff24b-5f3d-4d94-bce8-03706b5fcb0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:05.791133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-33fff24b-5f3d-4d94-bce8-03706b5fcb0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:05.868804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:05.901470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fff24b-5f3d-4d94-bce8-03706b5fcb0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:05.902939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fff24b-5f3d-4d94-bce8-03706b5fcb0b service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:05.921914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 664/1329] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:05 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:08.357918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f70e59-4a63-4599-9536-d6179dc10805 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:08.358448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f70e59-4a63-4599-9536-d6179dc10805 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:08.358448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f70e59-4a63-4599-9536-d6179dc10805 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:08.360011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 666/1330] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:08.365406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9df07-1f8c-490d-9e78-c2eff16a1f8c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:08.365852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9df07-1f8c-490d-9e78-c2eff16a1f8c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:08.365955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf9df07-1f8c-490d-9e78-c2eff16a1f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:08.432804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-8cf9df07-1f8c-490d-9e78-c2eff16a1f8c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:08.434004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8cf9df07-1f8c-490d-9e78-c2eff16a1f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:08.540323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 665/1331] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:08.576754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f70e59-4a63-4599-9536-d6179dc10805 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:08.579219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-73f70e59-4a63-4599-9536-d6179dc10805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:08.704723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f70e59-4a63-4599-9536-d6179dc10805 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:08.790682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146f91c-49a8-4b74-b7ad-c05ecfcd2d70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:08.791428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146f91c-49a8-4b74-b7ad-c05ecfcd2d70 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:08.791710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146f91c-49a8-4b74-b7ad-c05ecfcd2d70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:08.791984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146f91c-49a8-4b74-b7ad-c05ecfcd2d70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:08.793647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4146f91c-49a8-4b74-b7ad-c05ecfcd2d70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:08.822320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:08.910934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146f91c-49a8-4b74-b7ad-c05ecfcd2d70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:08.912417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146f91c-49a8-4b74-b7ad-c05ecfcd2d70 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:08.932309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 667/1332] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:05:11.209976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0c97c-c57a-4350-83a8-62ef18a4e60c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:11.210570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0c97c-c57a-4350-83a8-62ef18a4e60c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:11.210570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f0c97c-c57a-4350-83a8-62ef18a4e60c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:11.212743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 666/1333] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:11.218393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b47a8f-c73a-40c5-abb3-5a4047bb337e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:11.218792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b47a8f-c73a-40c5-abb3-5a4047bb337e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:11.218945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b47a8f-c73a-40c5-abb3-5a4047bb337e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:11.289854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-59b47a8f-c73a-40c5-abb3-5a4047bb337e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:11.291066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-59b47a8f-c73a-40c5-abb3-5a4047bb337e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:11.371591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:11.395814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 668/1334] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:11.436491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0c97c-c57a-4350-83a8-62ef18a4e60c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:11.438848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-74f0c97c-c57a-4350-83a8-62ef18a4e60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:11.484531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:11.552729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f0c97c-c57a-4350-83a8-62ef18a4e60c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:11.638695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f66f7-4537-4f3c-a131-160c0dced273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:11.639372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f66f7-4537-4f3c-a131-160c0dced273 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:11.639656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f66f7-4537-4f3c-a131-160c0dced273 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:11.639928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f66f7-4537-4f3c-a131-160c0dced273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:11.641566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-885f66f7-4537-4f3c-a131-160c0dced273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:11.727687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:11.765195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f66f7-4537-4f3c-a131-160c0dced273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:11.766727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f66f7-4537-4f3c-a131-160c0dced273 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:11.787287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 667/1335] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:14.292260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc2a32-97a2-4786-9e49-e762ce432748 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:14.292813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc2a32-97a2-4786-9e49-e762ce432748 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:14.292813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67dc2a32-97a2-4786-9e49-e762ce432748 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:14.294981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 669/1336] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:14.302752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705c588-5b43-4f59-af53-ce4cc29c2154 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:14.303233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705c588-5b43-4f59-af53-ce4cc29c2154 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:14.303359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e705c588-5b43-4f59-af53-ce4cc29c2154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:14.367754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e705c588-5b43-4f59-af53-ce4cc29c2154 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:14.369013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e705c588-5b43-4f59-af53-ce4cc29c2154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:14.472186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 668/1337] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:14 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:14.522605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf8fed-6484-40b9-bc0c-66e8e73c4641 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:14.523090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf8fed-6484-40b9-bc0c-66e8e73c4641 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:14.523505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf8fed-6484-40b9-bc0c-66e8e73c4641 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:14.525457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 670/1338] 10.222.0.69 () {54 vars in 964 bytes} [Fri May 20 09:05:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:14.533946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbaf675-0bd3-4978-9d6f-2dc1ad11e6ab None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:14.534228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbaf675-0bd3-4978-9d6f-2dc1ad11e6ab None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:14.534509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbaf675-0bd3-4978-9d6f-2dc1ad11e6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:14.589697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:14.612214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-fdbaf675-0bd3-4978-9d6f-2dc1ad11e6ab None None] MFA Rules not processed for user `349e99511c7c4307b27dc90b01f2dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:14.613435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fdbaf675-0bd3-4978-9d6f-2dc1ad11e6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:14.700419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 669/1339] 10.222.0.69 () {58 vars in 1061 bytes} [Fri May 20 09:05:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:05:14.714904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf8fed-6484-40b9-bc0c-66e8e73c4641 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:14.717503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9aaf8fed-6484-40b9-bc0c-66e8e73c4641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:14.846926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf8fed-6484-40b9-bc0c-66e8e73c4641 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:14.922973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9132cbfe-eba1-41d3-be12-05d0e4ce6d72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:14.923640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132cbfe-eba1-41d3-be12-05d0e4ce6d72 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:14.923923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132cbfe-eba1-41d3-be12-05d0e4ce6d72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:14.924199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9132cbfe-eba1-41d3-be12-05d0e4ce6d72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:14.925843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9132cbfe-eba1-41d3-be12-05d0e4ce6d72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:14.964615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:15.047628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132cbfe-eba1-41d3-be12-05d0e4ce6d72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:15.049780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9132cbfe-eba1-41d3-be12-05d0e4ce6d72 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:15.074705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 671/1340] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:19.820319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d8a0-2a3c-4a49-98a4-b148e2754dd9 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:19.820931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d8a0-2a3c-4a49-98a4-b148e2754dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:19.820931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7d8a0-2a3c-4a49-98a4-b148e2754dd9 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:19.822316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 670/1341] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:19.827685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb1ee6-1a97-434c-af7a-c36063dc45bd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:19.828132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb1ee6-1a97-434c-af7a-c36063dc45bd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:19.828234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb1ee6-1a97-434c-af7a-c36063dc45bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:19.899148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-7abb1ee6-1a97-434c-af7a-c36063dc45bd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:19.900457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7abb1ee6-1a97-434c-af7a-c36063dc45bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:19.994623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:20.008757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 672/1342] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:20.047432 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7d8a0-2a3c-4a49-98a4-b148e2754dd9 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:20.049814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-42f7d8a0-2a3c-4a49-98a4-b148e2754dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:20.159744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7d8a0-2a3c-4a49-98a4-b148e2754dd9 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:20.163688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4818c96-8e0d-4392-81bc-6a4d1fade0f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:20.164338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4818c96-8e0d-4392-81bc-6a4d1fade0f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:20.164614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4818c96-8e0d-4392-81bc-6a4d1fade0f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:20.164887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4818c96-8e0d-4392-81bc-6a4d1fade0f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:20.174457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4818c96-8e0d-4392-81bc-6a4d1fade0f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:20.175938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4818c96-8e0d-4392-81bc-6a4d1fade0f7 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:20.185628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 671/1343] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:23.002436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e4757-167a-4019-b04d-77af5a4e5f2c None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:23.002971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e4757-167a-4019-b04d-77af5a4e5f2c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:23.002971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e4757-167a-4019-b04d-77af5a4e5f2c None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:23.004430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 673/1344] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:23.009673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6755522-008b-4849-b996-e231e69517d6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:23.010115 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6755522-008b-4849-b996-e231e69517d6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:23.010242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6755522-008b-4849-b996-e231e69517d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:23.038094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b6755522-008b-4849-b996-e231e69517d6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:23.039292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b6755522-008b-4849-b996-e231e69517d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:23.067119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:23.122409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 672/1345] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:23 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:23.160755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e4757-167a-4019-b04d-77af5a4e5f2c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:23.172371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e4757-167a-4019-b04d-77af5a4e5f2c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:23.176209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebfa95-74b1-4316-99c7-6f2e11e86b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:23.176862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfa95-74b1-4316-99c7-6f2e11e86b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:23.177143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfa95-74b1-4316-99c7-6f2e11e86b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:23.177411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebfa95-74b1-4316-99c7-6f2e11e86b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:23.187220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebfa95-74b1-4316-99c7-6f2e11e86b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:23.188690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebfa95-74b1-4316-99c7-6f2e11e86b30 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:23.198802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 674/1346] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:23 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:27.290483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a72e4-e5e9-4bb7-9536-c75728a3bc02 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:27.291036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a72e4-e5e9-4bb7-9536-c75728a3bc02 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:27.291136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a72e4-e5e9-4bb7-9536-c75728a3bc02 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:27.292529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 673/1347] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:27.298217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c8a9-7d12-480b-a11b-65b88ee5197c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:27.298661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c8a9-7d12-480b-a11b-65b88ee5197c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:27.298754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275c8a9-7d12-480b-a11b-65b88ee5197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:27.317884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4275c8a9-7d12-480b-a11b-65b88ee5197c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:27.319067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4275c8a9-7d12-480b-a11b-65b88ee5197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:27.327787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 675/1348] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:05:27.365569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a72e4-e5e9-4bb7-9536-c75728a3bc02 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:27.377290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a72e4-e5e9-4bb7-9536-c75728a3bc02 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:27.381266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27528805-8dfa-40b4-a258-1286e5e3f577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:27.382135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528805-8dfa-40b4-a258-1286e5e3f577 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:27.382494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528805-8dfa-40b4-a258-1286e5e3f577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:27.382786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27528805-8dfa-40b4-a258-1286e5e3f577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:27.393967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528805-8dfa-40b4-a258-1286e5e3f577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:27.395460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27528805-8dfa-40b4-a258-1286e5e3f577 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:27.406273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 674/1349] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:32.665966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c93ab-472e-4f0f-b82a-064836d21530 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:32.666953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c93ab-472e-4f0f-b82a-064836d21530 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:32.667518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97c93ab-472e-4f0f-b82a-064836d21530 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:32.669690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 676/1350] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:05:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:32.674199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791bc8d-266d-4cd3-bce8-4ee93bd1f031 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:32.674589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791bc8d-266d-4cd3-bce8-4ee93bd1f031 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:32.674794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3791bc8d-266d-4cd3-bce8-4ee93bd1f031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:32.757200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3791bc8d-266d-4cd3-bce8-4ee93bd1f031 None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:32.758384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3791bc8d-266d-4cd3-bce8-4ee93bd1f031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:32.845298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 675/1351] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:05:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:37.165962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a001d-c7c9-4e3f-962d-1c0348c4d20c None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:37.166504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a001d-c7c9-4e3f-962d-1c0348c4d20c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:37.166504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629a001d-c7c9-4e3f-962d-1c0348c4d20c None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:37.168279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 677/1352] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:37.176351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93173ee-e5d8-4441-9b8e-3cd2e00a4acf None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:37.176914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93173ee-e5d8-4441-9b8e-3cd2e00a4acf None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:37.176914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93173ee-e5d8-4441-9b8e-3cd2e00a4acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:37.197459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a93173ee-e5d8-4441-9b8e-3cd2e00a4acf None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:37.198759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a93173ee-e5d8-4441-9b8e-3cd2e00a4acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:37.208157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 676/1353] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:05:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:37.312400 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=13 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:38.569137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c1b2-d26a-4d3f-a7a5-2fb179ec53ee None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:38.569808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c1b2-d26a-4d3f-a7a5-2fb179ec53ee None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:38.569808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1876c1b2-d26a-4d3f-a7a5-2fb179ec53ee None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:38.571362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 678/1354] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:38.576374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e023a-ec0f-4d34-bcd8-c56d846d49d8 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:38.576779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e023a-ec0f-4d34-bcd8-c56d846d49d8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:38.576923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e023a-ec0f-4d34-bcd8-c56d846d49d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:38.596040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-469e023a-ec0f-4d34-bcd8-c56d846d49d8 None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:38.597243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-469e023a-ec0f-4d34-bcd8-c56d846d49d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:38.606673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 677/1355] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:05:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:42.599102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64576df0-24ec-4f41-8d4e-40b04907e8a1 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:42.599676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64576df0-24ec-4f41-8d4e-40b04907e8a1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:42.599676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64576df0-24ec-4f41-8d4e-40b04907e8a1 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:42.600937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 679/1356] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:42.606259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8e57d-0f43-4e8e-b4e2-fae70bf8a11a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:42.606671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8e57d-0f43-4e8e-b4e2-fae70bf8a11a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:42.606800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b8e57d-0f43-4e8e-b4e2-fae70bf8a11a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:42.624405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a5b8e57d-0f43-4e8e-b4e2-fae70bf8a11a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:42.625583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a5b8e57d-0f43-4e8e-b4e2-fae70bf8a11a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:42.633470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 678/1357] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:42.687751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7febc-5246-4733-9e54-de1ea96b60ee None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:42.688041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7febc-5246-4733-9e54-de1ea96b60ee None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:42.688342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7febc-5246-4733-9e54-de1ea96b60ee None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:42.689743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 680/1358] 10.222.0.69 () {54 vars in 961 bytes} [Fri May 20 09:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:42.697628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c7ec8-8ca9-413e-97ce-8e3b6e7fe143 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:42.698216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c7ec8-8ca9-413e-97ce-8e3b6e7fe143 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:42.698404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5c7ec8-8ca9-413e-97ce-8e3b6e7fe143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:42.744960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:42.753616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3a5c7ec8-8ca9-413e-97ce-8e3b6e7fe143 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:42.754921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3a5c7ec8-8ca9-413e-97ce-8e3b6e7fe143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:42.847621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 679/1359] 10.222.0.69 () {58 vars in 1058 bytes} [Fri May 20 09:05:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:05:42.855793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7febc-5246-4733-9e54-de1ea96b60ee None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:42.872524 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7febc-5246-4733-9e54-de1ea96b60ee None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:42.879089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a09c60-a201-4af3-bdf7-7a3e06be7d64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:42.879875 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a09c60-a201-4af3-bdf7-7a3e06be7d64 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:42.880375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a09c60-a201-4af3-bdf7-7a3e06be7d64 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:42.880747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a09c60-a201-4af3-bdf7-7a3e06be7d64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:42.890190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a09c60-a201-4af3-bdf7-7a3e06be7d64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:42.892223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a09c60-a201-4af3-bdf7-7a3e06be7d64 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:42.902536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 681/1360] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:45.321114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10362b-2bd3-4391-b161-c2d12a215044 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:45.321692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10362b-2bd3-4391-b161-c2d12a215044 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:45.321692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10362b-2bd3-4391-b161-c2d12a215044 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:45.323001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 680/1361] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:45.329322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319f0a-b243-44d6-a1ed-bc6c948d64f7 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:45.329790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319f0a-b243-44d6-a1ed-bc6c948d64f7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:45.329872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319f0a-b243-44d6-a1ed-bc6c948d64f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:45.348160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8f319f0a-b243-44d6-a1ed-bc6c948d64f7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:45.349354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8f319f0a-b243-44d6-a1ed-bc6c948d64f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:45.357461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 682/1362] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:05:45.401020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10362b-2bd3-4391-b161-c2d12a215044 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:45.416110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10362b-2bd3-4391-b161-c2d12a215044 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:45.421706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005bf133-c062-4e7a-89d3-bc58d6bf7de6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:45.422540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bf133-c062-4e7a-89d3-bc58d6bf7de6 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:45.423028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bf133-c062-4e7a-89d3-bc58d6bf7de6 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:45.423516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005bf133-c062-4e7a-89d3-bc58d6bf7de6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:45.432487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bf133-c062-4e7a-89d3-bc58d6bf7de6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:45.434289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005bf133-c062-4e7a-89d3-bc58d6bf7de6 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:45.458845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 681/1363] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:47.980675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2db70-eb09-4d75-ac52-2d34ae365741 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:47.981261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2db70-eb09-4d75-ac52-2d34ae365741 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:47.981261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c2db70-eb09-4d75-ac52-2d34ae365741 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:47.982809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 683/1364] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:47.988150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37456f0-64dd-4242-aa99-5c805139f9f6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:47.988645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37456f0-64dd-4242-aa99-5c805139f9f6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:47.988838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37456f0-64dd-4242-aa99-5c805139f9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:48.007121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a37456f0-64dd-4242-aa99-5c805139f9f6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:48.008334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a37456f0-64dd-4242-aa99-5c805139f9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:48.018330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 682/1365] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:48.064553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b440b2-db84-426a-87bb-05da8d881ecb None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:48.064839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b440b2-db84-426a-87bb-05da8d881ecb None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:48.065120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b440b2-db84-426a-87bb-05da8d881ecb None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:48.066541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 684/1366] 10.222.0.69 () {54 vars in 961 bytes} [Fri May 20 09:05:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:48.072502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763a70c-c7a7-4bdb-8df7-3baa88378cc8 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:48.073578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763a70c-c7a7-4bdb-8df7-3baa88378cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:48.073965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b763a70c-c7a7-4bdb-8df7-3baa88378cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:48.104519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b763a70c-c7a7-4bdb-8df7-3baa88378cc8 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:48.105638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b763a70c-c7a7-4bdb-8df7-3baa88378cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:48.114884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 683/1367] 10.222.0.69 () {58 vars in 1058 bytes} [Fri May 20 09:05:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:48.123229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b440b2-db84-426a-87bb-05da8d881ecb None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:48.137266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b440b2-db84-426a-87bb-05da8d881ecb None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:48.142859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df102e5-0815-452e-b375-bdaaf184ca0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:48.143528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df102e5-0815-452e-b375-bdaaf184ca0d service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:48.143808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df102e5-0815-452e-b375-bdaaf184ca0d service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:48.144082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df102e5-0815-452e-b375-bdaaf184ca0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:48.152379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df102e5-0815-452e-b375-bdaaf184ca0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:48.153965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df102e5-0815-452e-b375-bdaaf184ca0d service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:48.163723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 685/1368] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:50.445889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed16662-9c58-40dc-9b42-fa59094b05b6 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:50.446850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed16662-9c58-40dc-9b42-fa59094b05b6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:50.447123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed16662-9c58-40dc-9b42-fa59094b05b6 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:50.448534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 684/1369] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:05:50.453994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2b322-c424-46f2-aa91-091eb5522e1e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:50.454499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2b322-c424-46f2-aa91-091eb5522e1e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:50.454715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b2b322-c424-46f2-aa91-091eb5522e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:50.474676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-69b2b322-c424-46f2-aa91-091eb5522e1e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:50.475949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-69b2b322-c424-46f2-aa91-091eb5522e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:50.487313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 686/1370] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:50.528503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed16662-9c58-40dc-9b42-fa59094b05b6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:50.548042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed16662-9c58-40dc-9b42-fa59094b05b6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:50.553472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c135dfb-e48c-4e48-b0cc-3a00a629a4de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:50.555478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c135dfb-e48c-4e48-b0cc-3a00a629a4de service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:50.555478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c135dfb-e48c-4e48-b0cc-3a00a629a4de service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:50.555478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c135dfb-e48c-4e48-b0cc-3a00a629a4de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:50.564605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c135dfb-e48c-4e48-b0cc-3a00a629a4de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:50.566428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c135dfb-e48c-4e48-b0cc-3a00a629a4de service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:50.577096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 685/1371] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:50.781476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3713-5414-4735-ad4b-3929e45c6514 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:50.781765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3713-5414-4735-ad4b-3929e45c6514 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:50.782043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce3713-5414-4735-ad4b-3929e45c6514 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:50.783879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 687/1372] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:50.790380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3f04-ac7a-4173-a29c-93da534bb0de None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:50.790838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3f04-ac7a-4173-a29c-93da534bb0de None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:50.791041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d3f04-ac7a-4173-a29c-93da534bb0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:50.810958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a15d3f04-ac7a-4173-a29c-93da534bb0de None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:50.812292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a15d3f04-ac7a-4173-a29c-93da534bb0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:50.821020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 686/1373] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:51.130483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bfb53-38e7-4e08-af43-d427d7c7da3a None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:51.130757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bfb53-38e7-4e08-af43-d427d7c7da3a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:51.131011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537bfb53-38e7-4e08-af43-d427d7c7da3a None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:51.132397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 688/1374] 10.222.0.69 () {54 vars in 963 bytes} [Fri May 20 09:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:51.137610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154958f2-18cf-47e0-9660-a425d2802b31 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:51.137897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154958f2-18cf-47e0-9660-a425d2802b31 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:51.138174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154958f2-18cf-47e0-9660-a425d2802b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:51.187767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-154958f2-18cf-47e0-9660-a425d2802b31 None None] MFA Rules not processed for user `4b672a2898b849338844b1c8e911646d`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:51.188974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-154958f2-18cf-47e0-9660-a425d2802b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:51.254674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 687/1375] 10.222.0.69 () {58 vars in 1060 bytes} [Fri May 20 09:05:51 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:51.262673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bfb53-38e7-4e08-af43-d427d7c7da3a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:51.274092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537bfb53-38e7-4e08-af43-d427d7c7da3a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:51.277928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8db2c-0648-410f-ac28-92341b03bce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:51.278667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8db2c-0648-410f-ac28-92341b03bce8 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:51.279017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8db2c-0648-410f-ac28-92341b03bce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:51.279369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8db2c-0648-410f-ac28-92341b03bce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:51.289722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8db2c-0648-410f-ac28-92341b03bce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:51.291546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d8db2c-0648-410f-ac28-92341b03bce8 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:51.302465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 689/1376] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:52.820464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04838978-5373-46a1-a99f-d5a48a00859f None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:52.821075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04838978-5373-46a1-a99f-d5a48a00859f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:52.821075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04838978-5373-46a1-a99f-d5a48a00859f None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:52.822515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 688/1377] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:52.828692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b5221-a36f-4cab-af27-b274824f5619 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:52.829333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b5221-a36f-4cab-af27-b274824f5619 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:52.829333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b5221-a36f-4cab-af27-b274824f5619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:52.849950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-756b5221-a36f-4cab-af27-b274824f5619 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:52.851181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-756b5221-a36f-4cab-af27-b274824f5619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:52.860556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 690/1378] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:52.909519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04838978-5373-46a1-a99f-d5a48a00859f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:52.925081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04838978-5373-46a1-a99f-d5a48a00859f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:52.930848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e5b96e-c413-482d-a4ed-8332eef4438b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:52.931628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5b96e-c413-482d-a4ed-8332eef4438b service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:52.931868 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5b96e-c413-482d-a4ed-8332eef4438b service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:52.932165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e5b96e-c413-482d-a4ed-8332eef4438b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:52.941674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5b96e-c413-482d-a4ed-8332eef4438b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:52.943389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e5b96e-c413-482d-a4ed-8332eef4438b service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:52.948014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:05:52.955094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 689/1379] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:55.222640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b742-e078-4e8e-88a6-9fd602cad5e4 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:55.223244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b742-e078-4e8e-88a6-9fd602cad5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:55.223244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9662b742-e078-4e8e-88a6-9fd602cad5e4 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:55.224565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 691/1380] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:55.230205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0cca2-2c70-410b-a870-c778cab0dbf2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:55.230667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0cca2-2c70-410b-a870-c778cab0dbf2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:55.230758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b0cca2-2c70-410b-a870-c778cab0dbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:55.249441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-80b0cca2-2c70-410b-a870-c778cab0dbf2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:55.250589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-80b0cca2-2c70-410b-a870-c778cab0dbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:55.258622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 690/1381] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:55.302438 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662b742-e078-4e8e-88a6-9fd602cad5e4 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:55.318145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9662b742-e078-4e8e-88a6-9fd602cad5e4 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:55.325614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc7ff1-6239-48e3-9458-5083e61e3ba0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:55.326395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc7ff1-6239-48e3-9458-5083e61e3ba0 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:55.326772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc7ff1-6239-48e3-9458-5083e61e3ba0 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:55.327174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc7ff1-6239-48e3-9458-5083e61e3ba0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:55.336240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc7ff1-6239-48e3-9458-5083e61e3ba0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:55.337988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cc7ff1-6239-48e3-9458-5083e61e3ba0 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:55.347916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 692/1382] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:57.609083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d995875-95eb-4635-9e40-23b56a272599 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:57.609668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d995875-95eb-4635-9e40-23b56a272599 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:57.609668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d995875-95eb-4635-9e40-23b56a272599 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:57.611236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 691/1383] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:57.616610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315c44-f381-4971-897c-18fd6a24a8a5 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:57.617091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315c44-f381-4971-897c-18fd6a24a8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:57.617196 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23315c44-f381-4971-897c-18fd6a24a8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:57.636281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-23315c44-f381-4971-897c-18fd6a24a8a5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:05:57.637464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-23315c44-f381-4971-897c-18fd6a24a8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:05:57.646430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 693/1384] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:05:57.690329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d995875-95eb-4635-9e40-23b56a272599 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:05:57.705202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d995875-95eb-4635-9e40-23b56a272599 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:05:57.710967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30fbcb9-6dbf-45c1-aac4-4e83957fed11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:05:57.711814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbcb9-6dbf-45c1-aac4-4e83957fed11 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:57.712264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbcb9-6dbf-45c1-aac4-4e83957fed11 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:57.712738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30fbcb9-6dbf-45c1-aac4-4e83957fed11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:57.721577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbcb9-6dbf-45c1-aac4-4e83957fed11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:05:57.723378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30fbcb9-6dbf-45c1-aac4-4e83957fed11 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:05:57.734310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 692/1385] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:05:59.973556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47497b1-0e47-44cd-ae50-f27799dd01a8 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:59.974134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47497b1-0e47-44cd-ae50-f27799dd01a8 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:59.974134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47497b1-0e47-44cd-ae50-f27799dd01a8 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:05:59.975532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 694/1386] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:05:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:05:59.980873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c743e6-6782-4932-b60d-c85d74d95a90 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:05:59.981430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c743e6-6782-4932-b60d-c85d74d95a90 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:05:59.981430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c743e6-6782-4932-b60d-c85d74d95a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:00.000601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c0c743e6-6782-4932-b60d-c85d74d95a90 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:00.001906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c0c743e6-6782-4932-b60d-c85d74d95a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:00.011304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 693/1387] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:05:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:00.057559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47497b1-0e47-44cd-ae50-f27799dd01a8 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:00.080568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47497b1-0e47-44cd-ae50-f27799dd01a8 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:00.087744 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f32a27-499c-45ea-9b69-07f08b57f939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:00.088662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f32a27-499c-45ea-9b69-07f08b57f939 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:00.088967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f32a27-499c-45ea-9b69-07f08b57f939 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:00.089430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f32a27-499c-45ea-9b69-07f08b57f939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:00.099610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f32a27-499c-45ea-9b69-07f08b57f939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:00.101275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f32a27-499c-45ea-9b69-07f08b57f939 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:00.112681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 695/1388] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:02.382692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b707-6af2-4a62-b6df-147728d43360 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:02.383277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b707-6af2-4a62-b6df-147728d43360 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:02.383277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5125b707-6af2-4a62-b6df-147728d43360 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:02.384737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 694/1389] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:06:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:06:02.390057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7ac6-9fff-4e43-b838-2e2640c1d3a0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:02.390490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7ac6-9fff-4e43-b838-2e2640c1d3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:02.390607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd7ac6-9fff-4e43-b838-2e2640c1d3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:02.409387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e1bd7ac6-9fff-4e43-b838-2e2640c1d3a0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:02.410594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e1bd7ac6-9fff-4e43-b838-2e2640c1d3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:02.420186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 696/1390] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:02.465755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125b707-6af2-4a62-b6df-147728d43360 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:02.480295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5125b707-6af2-4a62-b6df-147728d43360 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:02.485837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f3766-b8e7-41e6-8137-b5f4cd4541f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:02.486497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3766-b8e7-41e6-8137-b5f4cd4541f9 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:02.486731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3766-b8e7-41e6-8137-b5f4cd4541f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:02.487168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f3766-b8e7-41e6-8137-b5f4cd4541f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:02.495467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f3766-b8e7-41e6-8137-b5f4cd4541f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:02.497002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f3766-b8e7-41e6-8137-b5f4cd4541f9 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:02.506410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 695/1391] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:04.745771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce510-e50d-4699-b402-5c1ea0db4fcc None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:04.747328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce510-e50d-4699-b402-5c1ea0db4fcc None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:04.747921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce510-e50d-4699-b402-5c1ea0db4fcc None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:04.750090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 697/1392] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:06:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:06:04.758505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ab5f6-626e-401a-9e43-28315901bd76 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:04.759114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ab5f6-626e-401a-9e43-28315901bd76 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:04.759114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18ab5f6-626e-401a-9e43-28315901bd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:04.779387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f18ab5f6-626e-401a-9e43-28315901bd76 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:04.780696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f18ab5f6-626e-401a-9e43-28315901bd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:04.789281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 696/1393] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:04.842224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabce510-e50d-4699-b402-5c1ea0db4fcc None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:04.859732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabce510-e50d-4699-b402-5c1ea0db4fcc None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:04.865934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a1a7c8-8981-47f8-bbb5-5668d7ae3a44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:04.866584 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a7c8-8981-47f8-bbb5-5668d7ae3a44 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:04.866864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a7c8-8981-47f8-bbb5-5668d7ae3a44 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:04.867134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a1a7c8-8981-47f8-bbb5-5668d7ae3a44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:04.876138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1a7c8-8981-47f8-bbb5-5668d7ae3a44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:04.877740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a1a7c8-8981-47f8-bbb5-5668d7ae3a44 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:04.887459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 698/1394] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:07.012350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23744e-9d12-4479-bc44-7ca6131c9965 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.012924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23744e-9d12-4479-bc44-7ca6131c9965 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.012924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23744e-9d12-4479-bc44-7ca6131c9965 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.014291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 697/1395] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:07.019807 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b3de-4004-44e1-b545-4a15ca2a22b7 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.020267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b3de-4004-44e1-b545-4a15ca2a22b7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.020372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3425b3de-4004-44e1-b545-4a15ca2a22b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.039440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3425b3de-4004-44e1-b545-4a15ca2a22b7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:07.040784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3425b3de-4004-44e1-b545-4a15ca2a22b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:07.049543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 699/1396] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:07.104814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388802f-adfd-4a2c-ad68-0171cd29d8ed None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.105182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388802f-adfd-4a2c-ad68-0171cd29d8ed None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.105463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388802f-adfd-4a2c-ad68-0171cd29d8ed None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.106849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 698/1397] 10.222.0.69 () {54 vars in 962 bytes} [Fri May 20 09:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:07.112118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13559-e805-437d-a5d7-b1c3c2418b0b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.112409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13559-e805-437d-a5d7-b1c3c2418b0b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.112683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c13559-e805-437d-a5d7-b1c3c2418b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.113555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55f35-d0fe-442d-b467-822f699a39de None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.113846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55f35-d0fe-442d-b467-822f699a39de None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.114152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d55f35-d0fe-442d-b467-822f699a39de None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.115450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 699/1398] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:07.120398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e398f8-6d7a-4efd-b9a4-7ac2350be4d6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.120684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e398f8-6d7a-4efd-b9a4-7ac2350be4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.120961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e398f8-6d7a-4efd-b9a4-7ac2350be4d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.138683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e3e398f8-6d7a-4efd-b9a4-7ac2350be4d6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:07.139873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e3e398f8-6d7a-4efd-b9a4-7ac2350be4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:07.147662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 700/1399] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:06:07.163842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f1c13559-e805-437d-a5d7-b1c3c2418b0b None None] MFA Rules not processed for user `68d25a45f9ee4ae58c6751a8781d12d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:07.165038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f1c13559-e805-437d-a5d7-b1c3c2418b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:07.192431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e398f8-6d7a-4efd-b9a4-7ac2350be4d6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:07.204622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e398f8-6d7a-4efd-b9a4-7ac2350be4d6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:07.210178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9f4f3-c473-4869-a2a5-16567f9cadb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:07.210818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9f4f3-c473-4869-a2a5-16567f9cadb4 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.211223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9f4f3-c473-4869-a2a5-16567f9cadb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.211463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9f4f3-c473-4869-a2a5-16567f9cadb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.219684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9f4f3-c473-4869-a2a5-16567f9cadb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:07.221288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9f4f3-c473-4869-a2a5-16567f9cadb4 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:07.223213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 700/1400] 10.222.0.69 () {58 vars in 1059 bytes} [Fri May 20 09:06:07 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:07.230932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 701/1401] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:06:07.231465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c13559-e805-437d-a5d7-b1c3c2418b0b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:07.239232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c13559-e805-437d-a5d7-b1c3c2418b0b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:07.243675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55bc368-1dbc-48c0-bb7f-b47cda3a619d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d25a45f9ee4ae58c6751a8781d12d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:07.244329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bc368-1dbc-48c0-bb7f-b47cda3a619d service glance] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.244607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bc368-1dbc-48c0-bb7f-b47cda3a619d service glance] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.244884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55bc368-1dbc-48c0-bb7f-b47cda3a619d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.254448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55bc368-1dbc-48c0-bb7f-b47cda3a619d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:07.256111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55bc368-1dbc-48c0-bb7f-b47cda3a619d service glance] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:07.269097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 701/1402] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:07.807709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523c74d-57d4-453d-9bf4-b6ed49a82381 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.808083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523c74d-57d4-453d-9bf4-b6ed49a82381 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.808423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523c74d-57d4-453d-9bf4-b6ed49a82381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.857231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-5523c74d-57d4-453d-9bf4-b6ed49a82381 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:07.858459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5523c74d-57d4-453d-9bf4-b6ed49a82381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:07.923077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 702/1403] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:06:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:07.963790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4deb93-262c-41ed-b59f-020af331fa14 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.964077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4deb93-262c-41ed-b59f-020af331fa14 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.964347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4deb93-262c-41ed-b59f-020af331fa14 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:07.965727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 702/1404] 10.222.0.69 () {54 vars in 942 bytes} [Fri May 20 09:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:07.972268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96784-441b-4183-9d5b-21ce7e4d393f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:07.972556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96784-441b-4183-9d5b-21ce7e4d393f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:07.972832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa96784-441b-4183-9d5b-21ce7e4d393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:08.023603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ffa96784-441b-4183-9d5b-21ce7e4d393f None None] MFA Rules not processed for user `0c8203a8a7464c3bac93e5d42808c00f`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:08.024814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ffa96784-441b-4183-9d5b-21ce7e4d393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:08.059430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:06:08.096896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 703/1405] 10.222.0.69 () {58 vars in 1039 bytes} [Fri May 20 09:06:07 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:08.106457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4deb93-262c-41ed-b59f-020af331fa14 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:08.114060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:06:08.121185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4deb93-262c-41ed-b59f-020af331fa14 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:08.126172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b588dfb1-5864-4a43-b9f5-cdcd93d92438 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:08.126808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588dfb1-5864-4a43-b9f5-cdcd93d92438 service swift] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:08.127082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588dfb1-5864-4a43-b9f5-cdcd93d92438 service swift] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:08.127370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b588dfb1-5864-4a43-b9f5-cdcd93d92438 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:08.135625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588dfb1-5864-4a43-b9f5-cdcd93d92438 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:08.137209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b588dfb1-5864-4a43-b9f5-cdcd93d92438 service swift] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:08.145078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 703/1406] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:06:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:06:09.457769 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b02b8-4a38-4a12-b791-12c3b322536e None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:09.458349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b02b8-4a38-4a12-b791-12c3b322536e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:09.458349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78b02b8-4a38-4a12-b791-12c3b322536e None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:09.459982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 704/1407] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:09.464907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b733a-eb03-4249-906f-ea256b6c1f55 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:09.465685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b733a-eb03-4249-906f-ea256b6c1f55 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:09.466130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b733a-eb03-4249-906f-ea256b6c1f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:09.486107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-597b733a-eb03-4249-906f-ea256b6c1f55 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:09.487331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-597b733a-eb03-4249-906f-ea256b6c1f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:09.496652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 704/1408] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:09.548795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b02b8-4a38-4a12-b791-12c3b322536e None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:09.565562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78b02b8-4a38-4a12-b791-12c3b322536e None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:09.570901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f5193b-1b5c-4074-b234-faccdc285882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:09.571684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5193b-1b5c-4074-b234-faccdc285882 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:09.571984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5193b-1b5c-4074-b234-faccdc285882 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:09.572274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f5193b-1b5c-4074-b234-faccdc285882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:09.582667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5193b-1b5c-4074-b234-faccdc285882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:09.584325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f5193b-1b5c-4074-b234-faccdc285882 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:09.594201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 705/1409] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:19.481413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=8 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:06:19.592371 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=21 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:06:32.381392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf3536-33d0-45da-a979-23531007c234 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:32.382052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf3536-33d0-45da-a979-23531007c234 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:32.382052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acf3536-33d0-45da-a979-23531007c234 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:32.383405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 705/1410] 10.222.0.69 () {54 vars in 918 bytes} [Fri May 20 09:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:32.389487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1af88-a33c-4013-aa2c-3ffd87f7136f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:32.389972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1af88-a33c-4013-aa2c-3ffd87f7136f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:32.390081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1af88-a33c-4013-aa2c-3ffd87f7136f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:32.413083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b7e1af88-a33c-4013-aa2c-3ffd87f7136f None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:32.414294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b7e1af88-a33c-4013-aa2c-3ffd87f7136f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:32.423793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 706/1411] 10.222.0.69 () {58 vars in 1015 bytes} [Fri May 20 09:06:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:32.438218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1036de7-6701-4376-a020-319ff115586a None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:32.438504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1036de7-6701-4376-a020-319ff115586a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:32.438787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1036de7-6701-4376-a020-319ff115586a None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:32.440228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 706/1412] 10.222.0.69 () {54 vars in 918 bytes} [Fri May 20 09:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:32.444827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f241fd9-4b2b-4859-a874-c3b493b76dd7 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:32.445111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f241fd9-4b2b-4859-a874-c3b493b76dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:32.445386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f241fd9-4b2b-4859-a874-c3b493b76dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:32.464145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-2f241fd9-4b2b-4859-a874-c3b493b76dd7 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:32.465317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2f241fd9-4b2b-4859-a874-c3b493b76dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:32.475048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 707/1413] 10.222.0.69 () {58 vars in 1015 bytes} [Fri May 20 09:06:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:06:35.148385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584790ed-345a-45fd-b582-53cd12b6910a None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:35.148949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584790ed-345a-45fd-b582-53cd12b6910a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:35.148949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584790ed-345a-45fd-b582-53cd12b6910a None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:35.151136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:06:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:35.158231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b9e5-38c1-4696-928f-089fc5317ced None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:35.158637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b9e5-38c1-4696-928f-089fc5317ced None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:35.158772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88b9e5-38c1-4696-928f-089fc5317ced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:35.185166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-2f88b9e5-38c1-4696-928f-089fc5317ced None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:35.186378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2f88b9e5-38c1-4696-928f-089fc5317ced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:35.196350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:06:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:35.245740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584790ed-345a-45fd-b582-53cd12b6910a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:35.262489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584790ed-345a-45fd-b582-53cd12b6910a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:35.268967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0fcd5-0e97-4332-87b4-7b6a806a4954 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:35.269603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0fcd5-0e97-4332-87b4-7b6a806a4954 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:35.269885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0fcd5-0e97-4332-87b4-7b6a806a4954 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:35.270159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0fcd5-0e97-4332-87b4-7b6a806a4954 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:35.278720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0fcd5-0e97-4332-87b4-7b6a806a4954 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:35.280346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0fcd5-0e97-4332-87b4-7b6a806a4954 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:35.290310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 708/1416] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:36.279283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd911-fd5e-48d0-8df0-a31daf0a70a4 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:36.279878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd911-fd5e-48d0-8df0-a31daf0a70a4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:36.279878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cd911-fd5e-48d0-8df0-a31daf0a70a4 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:36.281227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:36.287954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965890d0-6692-4802-8cab-85b5312ea31f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:36.288702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965890d0-6692-4802-8cab-85b5312ea31f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:36.288865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965890d0-6692-4802-8cab-85b5312ea31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:36.309079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-965890d0-6692-4802-8cab-85b5312ea31f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:36.310285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-965890d0-6692-4802-8cab-85b5312ea31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:36.319712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:36.495285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cd911-fd5e-48d0-8df0-a31daf0a70a4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:36.512083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cd911-fd5e-48d0-8df0-a31daf0a70a4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:36.518464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a24ee-62be-4418-99c0-10981f607b42 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:36.519096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a24ee-62be-4418-99c0-10981f607b42 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:36.519407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a24ee-62be-4418-99c0-10981f607b42 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:36.519682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a24ee-62be-4418-99c0-10981f607b42 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:36.528917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a24ee-62be-4418-99c0-10981f607b42 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:36.530569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a24ee-62be-4418-99c0-10981f607b42 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:36.540817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 710/1419] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:37.480206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45d9e9-fae2-4a32-86c4-438fcc9a8f36 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:37.481097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45d9e9-fae2-4a32-86c4-438fcc9a8f36 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:37.481097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45d9e9-fae2-4a32-86c4-438fcc9a8f36 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:37.483170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:06:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:37.491365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11699eef-9408-4f34-9bb6-447bc46274a3 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:37.491794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11699eef-9408-4f34-9bb6-447bc46274a3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:37.492026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11699eef-9408-4f34-9bb6-447bc46274a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:37.512388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-11699eef-9408-4f34-9bb6-447bc46274a3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:37.513560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-11699eef-9408-4f34-9bb6-447bc46274a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:37.521801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:37.671922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45d9e9-fae2-4a32-86c4-438fcc9a8f36 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:37.692516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45d9e9-fae2-4a32-86c4-438fcc9a8f36 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:37.699301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f552e-2d1e-4f2c-b50b-53c845cbbe3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:37.699947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f552e-2d1e-4f2c-b50b-53c845cbbe3d service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:37.700230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f552e-2d1e-4f2c-b50b-53c845cbbe3d service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:37.700480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f552e-2d1e-4f2c-b50b-53c845cbbe3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:37.710487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f552e-2d1e-4f2c-b50b-53c845cbbe3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:37.712123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f552e-2d1e-4f2c-b50b-53c845cbbe3d service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:37.724478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 711/1422] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:38.676846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c620-0cad-4d3b-8b75-ae613c63b891 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:38.677540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c620-0cad-4d3b-8b75-ae613c63b891 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:38.677540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34c620-0cad-4d3b-8b75-ae613c63b891 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:38.683215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:06:38 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:38.691209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b553788-2d98-4846-b8fd-a34c1de2a513 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:38.691741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b553788-2d98-4846-b8fd-a34c1de2a513 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:38.691854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b553788-2d98-4846-b8fd-a34c1de2a513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:38.712996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3b553788-2d98-4846-b8fd-a34c1de2a513 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:38.714208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3b553788-2d98-4846-b8fd-a34c1de2a513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:38.726318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:38.882300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34c620-0cad-4d3b-8b75-ae613c63b891 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:38.905588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34c620-0cad-4d3b-8b75-ae613c63b891 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:38.917028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67317756-cdb8-44ff-8784-b96534a62489 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:38.917679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317756-cdb8-44ff-8784-b96534a62489 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:38.917963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317756-cdb8-44ff-8784-b96534a62489 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:38.918238 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67317756-cdb8-44ff-8784-b96534a62489 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:38.928188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67317756-cdb8-44ff-8784-b96534a62489 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:38.929735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67317756-cdb8-44ff-8784-b96534a62489 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:38.943732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 713/1425] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:39.889790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9039d2-6a18-4041-b87d-4c299e4f0af7 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:39.890360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9039d2-6a18-4041-b87d-4c299e4f0af7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:39.890360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9039d2-6a18-4041-b87d-4c299e4f0af7 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:39.891777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:39.897989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7e483-658b-4c81-b398-0b005c44b92b None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:39.898734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7e483-658b-4c81-b398-0b005c44b92b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:39.898881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7e483-658b-4c81-b398-0b005c44b92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:39.919683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f7b7e483-658b-4c81-b398-0b005c44b92b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:39.920900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f7b7e483-658b-4c81-b398-0b005c44b92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:39.931028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:40.082358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9039d2-6a18-4041-b87d-4c299e4f0af7 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:40.098786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9039d2-6a18-4041-b87d-4c299e4f0af7 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:40.105444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e07e2-22ce-43f6-90be-22134ca871ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:40.106065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e07e2-22ce-43f6-90be-22134ca871ee service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:40.106348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e07e2-22ce-43f6-90be-22134ca871ee service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:40.106613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e07e2-22ce-43f6-90be-22134ca871ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:40.114858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e07e2-22ce-43f6-90be-22134ca871ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:40.116473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e07e2-22ce-43f6-90be-22134ca871ee service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:40.125877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 714/1428] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:40.990194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751140a4-754d-4e58-a7c6-03506587d189 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:40.990789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751140a4-754d-4e58-a7c6-03506587d189 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:40.990789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751140a4-754d-4e58-a7c6-03506587d189 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:40.992147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:41.000636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067fb0e-f8a7-453a-a2ec-7400bae4181b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:41.001049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067fb0e-f8a7-453a-a2ec-7400bae4181b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:41.001166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067fb0e-f8a7-453a-a2ec-7400bae4181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:41.023416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3067fb0e-f8a7-453a-a2ec-7400bae4181b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:41.024625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3067fb0e-f8a7-453a-a2ec-7400bae4181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:41.036718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:41.187657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751140a4-754d-4e58-a7c6-03506587d189 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:41.210423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751140a4-754d-4e58-a7c6-03506587d189 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:41.216294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4ebcc-b4f7-4ba5-b9d6-a98b52be7c2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:41.216968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4ebcc-b4f7-4ba5-b9d6-a98b52be7c2d service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:41.217255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4ebcc-b4f7-4ba5-b9d6-a98b52be7c2d service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:41.217537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4ebcc-b4f7-4ba5-b9d6-a98b52be7c2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:41.226170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4ebcc-b4f7-4ba5-b9d6-a98b52be7c2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:41.227810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a4ebcc-b4f7-4ba5-b9d6-a98b52be7c2d service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:41.237767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 716/1431] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:42.108786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33e807-f059-4673-965d-10021d8c7903 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:42.109315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33e807-f059-4673-965d-10021d8c7903 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:42.109315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc33e807-f059-4673-965d-10021d8c7903 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:42.111381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:06:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:42.118937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a67576-912b-4a9d-a6fb-5ebfbe5ba788 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:42.119403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a67576-912b-4a9d-a6fb-5ebfbe5ba788 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:42.119563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a67576-912b-4a9d-a6fb-5ebfbe5ba788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:42.137809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f4a67576-912b-4a9d-a6fb-5ebfbe5ba788 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:42.138986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f4a67576-912b-4a9d-a6fb-5ebfbe5ba788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:42.147602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:42.294983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33e807-f059-4673-965d-10021d8c7903 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:42.313908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc33e807-f059-4673-965d-10021d8c7903 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:42.320322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8227bf13-bbf9-4902-a2f0-e6130561be82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:42.320949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227bf13-bbf9-4902-a2f0-e6130561be82 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:42.321228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227bf13-bbf9-4902-a2f0-e6130561be82 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:42.321497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8227bf13-bbf9-4902-a2f0-e6130561be82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:42.329899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227bf13-bbf9-4902-a2f0-e6130561be82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:42.331508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8227bf13-bbf9-4902-a2f0-e6130561be82 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:42.340872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 717/1434] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:43.210731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e9a9-1911-41b1-9fe8-a7160d83bd18 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:43.211352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e9a9-1911-41b1-9fe8-a7160d83bd18 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:43.211352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7e9a9-1911-41b1-9fe8-a7160d83bd18 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:43.213478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:06:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:43.222055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51fc0f-4934-4bd0-a3f8-9813487c3497 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:43.222789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51fc0f-4934-4bd0-a3f8-9813487c3497 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:43.223101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a51fc0f-4934-4bd0-a3f8-9813487c3497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:43.244412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6a51fc0f-4934-4bd0-a3f8-9813487c3497 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:43.245599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6a51fc0f-4934-4bd0-a3f8-9813487c3497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:43.254805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:43.404756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7e9a9-1911-41b1-9fe8-a7160d83bd18 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:43.426798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7e9a9-1911-41b1-9fe8-a7160d83bd18 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:43.433375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920fb41-d4b9-4acb-9698-b93ac8a1d6e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:43.434007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920fb41-d4b9-4acb-9698-b93ac8a1d6e2 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:43.434286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920fb41-d4b9-4acb-9698-b93ac8a1d6e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:43.434557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920fb41-d4b9-4acb-9698-b93ac8a1d6e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:43.442764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920fb41-d4b9-4acb-9698-b93ac8a1d6e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:43.444447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920fb41-d4b9-4acb-9698-b93ac8a1d6e2 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:43.454156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 719/1437] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:44.330161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7fe5-8919-44ac-9765-16eb00ca72d0 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:44.330984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7fe5-8919-44ac-9765-16eb00ca72d0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:44.330984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7fe5-8919-44ac-9765-16eb00ca72d0 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:44.333191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:06:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:44.339173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0810ad-c646-431a-aaa3-93418146eedd None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:44.339651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0810ad-c646-431a-aaa3-93418146eedd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:44.339756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0810ad-c646-431a-aaa3-93418146eedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:44.359621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-fa0810ad-c646-431a-aaa3-93418146eedd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:44.360824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fa0810ad-c646-431a-aaa3-93418146eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:44.370435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:44.525234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f7fe5-8919-44ac-9765-16eb00ca72d0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:44.551128 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f7fe5-8919-44ac-9765-16eb00ca72d0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:44.558519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613fadc-8094-479d-b0a3-c42f5c8c5dc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:44.559158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613fadc-8094-479d-b0a3-c42f5c8c5dc5 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:44.559481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613fadc-8094-479d-b0a3-c42f5c8c5dc5 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:44.559774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613fadc-8094-479d-b0a3-c42f5c8c5dc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:44.568721 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613fadc-8094-479d-b0a3-c42f5c8c5dc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:44.570405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613fadc-8094-479d-b0a3-c42f5c8c5dc5 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:44.580341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 720/1440] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:46.660571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88a274-fec7-4b42-9e1d-1af454a9bfed None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:46.661498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88a274-fec7-4b42-9e1d-1af454a9bfed None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:46.661498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca88a274-fec7-4b42-9e1d-1af454a9bfed None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:46.663690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:06:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:46.670232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d637e-6b74-4205-b471-a0eb93c43602 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:46.670928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d637e-6b74-4205-b471-a0eb93c43602 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:46.671127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2d637e-6b74-4205-b471-a0eb93c43602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:46.691297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4c2d637e-6b74-4205-b471-a0eb93c43602 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:46.692487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4c2d637e-6b74-4205-b471-a0eb93c43602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:46.700889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:46.740105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88a274-fec7-4b42-9e1d-1af454a9bfed None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:46.755756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca88a274-fec7-4b42-9e1d-1af454a9bfed None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:46.761663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6d767d-c47d-4ed8-b5a1-78611ade7672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:46.762304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d767d-c47d-4ed8-b5a1-78611ade7672 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:46.762578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d767d-c47d-4ed8-b5a1-78611ade7672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:46.762842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6d767d-c47d-4ed8-b5a1-78611ade7672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:46.776929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d767d-c47d-4ed8-b5a1-78611ade7672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:46.778404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6d767d-c47d-4ed8-b5a1-78611ade7672 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:46.789082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 722/1443] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:49.155870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47e773-1438-4315-9fcc-b3ace22d3469 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:49.156496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47e773-1438-4315-9fcc-b3ace22d3469 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:49.156496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47e773-1438-4315-9fcc-b3ace22d3469 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:49.158142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:06:49.164779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6828f-c262-4d75-aa38-445adfc46d66 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:49.165546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6828f-c262-4d75-aa38-445adfc46d66 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:49.165876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6828f-c262-4d75-aa38-445adfc46d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:49.185330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-edc6828f-c262-4d75-aa38-445adfc46d66 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:49.186535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-edc6828f-c262-4d75-aa38-445adfc46d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:49.195700 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:49.234212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47e773-1438-4315-9fcc-b3ace22d3469 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:49.252043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47e773-1438-4315-9fcc-b3ace22d3469 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:49.257864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711370e4-acab-4bc0-badd-fb9b6db8cfe7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:49.258504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711370e4-acab-4bc0-badd-fb9b6db8cfe7 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:49.258820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711370e4-acab-4bc0-badd-fb9b6db8cfe7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:49.259114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711370e4-acab-4bc0-badd-fb9b6db8cfe7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:49.269538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711370e4-acab-4bc0-badd-fb9b6db8cfe7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:49.270967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711370e4-acab-4bc0-badd-fb9b6db8cfe7 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:49.280782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 723/1446] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:51.788129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9649-0a9e-4ee1-be01-9cfcfe961548 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:51.788595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9649-0a9e-4ee1-be01-9cfcfe961548 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:51.788713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab9649-0a9e-4ee1-be01-9cfcfe961548 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:51.790084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:06:51.796368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede8534-40c7-4161-b4b3-9cea85d4f213 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:51.796864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede8534-40c7-4161-b4b3-9cea85d4f213 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:51.797351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede8534-40c7-4161-b4b3-9cea85d4f213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:51.823184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1ede8534-40c7-4161-b4b3-9cea85d4f213 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:51.825054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1ede8534-40c7-4161-b4b3-9cea85d4f213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:51.839779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:51.881871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9649-0a9e-4ee1-be01-9cfcfe961548 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:51.901474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab9649-0a9e-4ee1-be01-9cfcfe961548 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:51.906070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ebe20-2f13-4bd7-a278-b90ea94d39bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:51.906716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ebe20-2f13-4bd7-a278-b90ea94d39bd service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:51.907000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ebe20-2f13-4bd7-a278-b90ea94d39bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:51.907631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ebe20-2f13-4bd7-a278-b90ea94d39bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:51.917696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ebe20-2f13-4bd7-a278-b90ea94d39bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:51.919158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ebe20-2f13-4bd7-a278-b90ea94d39bd service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:51.928929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 725/1449] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:54.344140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e8ea8-d5e0-421d-868e-8c79eaf4b99b None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:54.345062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e8ea8-d5e0-421d-868e-8c79eaf4b99b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:54.345062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4e8ea8-d5e0-421d-868e-8c79eaf4b99b None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:54.347086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:06:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:54.352595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98155c29-9501-4ef4-ae19-5d7a634fc654 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:54.353046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98155c29-9501-4ef4-ae19-5d7a634fc654 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:54.353151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98155c29-9501-4ef4-ae19-5d7a634fc654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:54.371957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-98155c29-9501-4ef4-ae19-5d7a634fc654 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:54.373153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-98155c29-9501-4ef4-ae19-5d7a634fc654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:54.381813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:54.425721 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e8ea8-d5e0-421d-868e-8c79eaf4b99b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:54.444771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4e8ea8-d5e0-421d-868e-8c79eaf4b99b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:54.450443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce10229-22fc-4b1b-ab50-38c31e6e9eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:54.451087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce10229-22fc-4b1b-ab50-38c31e6e9eac service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:54.451409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce10229-22fc-4b1b-ab50-38c31e6e9eac service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:54.451691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce10229-22fc-4b1b-ab50-38c31e6e9eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:54.461015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce10229-22fc-4b1b-ab50-38c31e6e9eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:54.462444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce10229-22fc-4b1b-ab50-38c31e6e9eac service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:54.472269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 726/1452] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:56.734302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90ee55-d4f9-416c-9aab-2611bcbde86c None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:56.734907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90ee55-d4f9-416c-9aab-2611bcbde86c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:56.734907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a90ee55-d4f9-416c-9aab-2611bcbde86c None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:56.736562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:56.742718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120dcf-0d23-4ca7-9dcc-7560e62a60a3 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:56.743178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120dcf-0d23-4ca7-9dcc-7560e62a60a3 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:56.743327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df120dcf-0d23-4ca7-9dcc-7560e62a60a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:56.765368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-df120dcf-0d23-4ca7-9dcc-7560e62a60a3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:56.766559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-df120dcf-0d23-4ca7-9dcc-7560e62a60a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:56.775842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:56.817854 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90ee55-d4f9-416c-9aab-2611bcbde86c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:56.837348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a90ee55-d4f9-416c-9aab-2611bcbde86c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:56.842227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db892e3-e1db-4991-96e5-e39f39dfca9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:56.842842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db892e3-e1db-4991-96e5-e39f39dfca9c service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:56.843121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db892e3-e1db-4991-96e5-e39f39dfca9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:56.843433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db892e3-e1db-4991-96e5-e39f39dfca9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:56.852910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db892e3-e1db-4991-96e5-e39f39dfca9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:56.854360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db892e3-e1db-4991-96e5-e39f39dfca9c service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:56.863918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 728/1455] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:06:59.093642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0108016-2642-4e5f-8506-02867d2fc9b8 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:59.094230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0108016-2642-4e5f-8506-02867d2fc9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:59.094230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0108016-2642-4e5f-8506-02867d2fc9b8 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:59.095490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:06:59.101646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862665ac-f5b3-44bb-9abe-5c2e9c354702 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:59.102097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862665ac-f5b3-44bb-9abe-5c2e9c354702 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:59.102271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862665ac-f5b3-44bb-9abe-5c2e9c354702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:59.126399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-862665ac-f5b3-44bb-9abe-5c2e9c354702 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:06:59.127612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-862665ac-f5b3-44bb-9abe-5c2e9c354702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:06:59.135989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:06:59.173557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0108016-2642-4e5f-8506-02867d2fc9b8 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:06:59.185504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0108016-2642-4e5f-8506-02867d2fc9b8 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:06:59.189742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370a690c-3b0e-4f2c-a9f5-9dd509945d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:06:59.190366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a690c-3b0e-4f2c-a9f5-9dd509945d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:06:59.190639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a690c-3b0e-4f2c-a9f5-9dd509945d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:06:59.190907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370a690c-3b0e-4f2c-a9f5-9dd509945d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:06:59.202236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a690c-3b0e-4f2c-a9f5-9dd509945d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:06:59.203688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370a690c-3b0e-4f2c-a9f5-9dd509945d34 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:06:59.215659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 729/1458] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:01.460177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b36a6b-3c32-426c-bc51-29f2d29f8390 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:01.460781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b36a6b-3c32-426c-bc51-29f2d29f8390 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:01.460781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b36a6b-3c32-426c-bc51-29f2d29f8390 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:01.462103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:01.468593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf2654-a0d5-40b7-8164-fe092c0b50cd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:01.469034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf2654-a0d5-40b7-8164-fe092c0b50cd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:01.469162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf2654-a0d5-40b7-8164-fe092c0b50cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:01.478830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=70 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:07:01.497425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-58bf2654-a0d5-40b7-8164-fe092c0b50cd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:01.498637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-58bf2654-a0d5-40b7-8164-fe092c0b50cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:01.507126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:01.543915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b36a6b-3c32-426c-bc51-29f2d29f8390 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:01.550102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=73 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:07:01.556712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b36a6b-3c32-426c-bc51-29f2d29f8390 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:01.560419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9869b-0e88-4c2f-ab7a-9e4eaa73f3ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:01.561048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9869b-0e88-4c2f-ab7a-9e4eaa73f3ba service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:01.561322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9869b-0e88-4c2f-ab7a-9e4eaa73f3ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:01.561596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9869b-0e88-4c2f-ab7a-9e4eaa73f3ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:01.570828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9869b-0e88-4c2f-ab7a-9e4eaa73f3ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:01.572321 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9869b-0e88-4c2f-ab7a-9e4eaa73f3ba service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:01.581841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 731/1461] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:07:03.833595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf9f12-4ed5-4630-a535-788d819b3f3d None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:03.834213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf9f12-4ed5-4630-a535-788d819b3f3d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:03.834213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf9f12-4ed5-4630-a535-788d819b3f3d None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:03.836614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:07:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:03.845648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172192-6574-45c8-abd9-532b09468ca3 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:03.846060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172192-6574-45c8-abd9-532b09468ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:03.846240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05172192-6574-45c8-abd9-532b09468ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:03.873365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-05172192-6574-45c8-abd9-532b09468ca3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:03.874605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-05172192-6574-45c8-abd9-532b09468ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:03.883646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:03.922062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf9f12-4ed5-4630-a535-788d819b3f3d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:03.937162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf9f12-4ed5-4630-a535-788d819b3f3d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:03.942069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2c362b-448d-427e-b8c4-5fb387fe11bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:03.942708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c362b-448d-427e-b8c4-5fb387fe11bf service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:03.942985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c362b-448d-427e-b8c4-5fb387fe11bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:03.943237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2c362b-448d-427e-b8c4-5fb387fe11bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:03.953518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c362b-448d-427e-b8c4-5fb387fe11bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:03.954950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2c362b-448d-427e-b8c4-5fb387fe11bf service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:03.965465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 732/1464] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:06.209303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a70b-aa61-469a-bd29-45f7593d1624 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:06.210143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a70b-aa61-469a-bd29-45f7593d1624 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:06.211677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e693a70b-aa61-469a-bd29-45f7593d1624 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:06.213933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:07:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:06.220960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cceff-5323-49b4-8476-842761c0546b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:06.221731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cceff-5323-49b4-8476-842761c0546b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:06.221875 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103cceff-5323-49b4-8476-842761c0546b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:06.242965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-103cceff-5323-49b4-8476-842761c0546b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:06.244194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-103cceff-5323-49b4-8476-842761c0546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:06.252693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:06.289534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a70b-aa61-469a-bd29-45f7593d1624 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:06.302524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e693a70b-aa61-469a-bd29-45f7593d1624 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:06.306853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bd002-543b-48b3-9793-300ec99bc9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:06.307574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bd002-543b-48b3-9793-300ec99bc9ba service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:06.307857 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bd002-543b-48b3-9793-300ec99bc9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:06.308135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bd002-543b-48b3-9793-300ec99bc9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:06.317431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bd002-543b-48b3-9793-300ec99bc9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:06.318872 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bd002-543b-48b3-9793-300ec99bc9ba service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:06.328615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 734/1467] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:08.526147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fbab5-486a-4ee8-8f47-1a0f8c6767bf None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:08.526760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fbab5-486a-4ee8-8f47-1a0f8c6767bf None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:08.526760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fbab5-486a-4ee8-8f47-1a0f8c6767bf None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:08.528109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:08.533994 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeda72d-51a0-48ad-8743-ad3808f44102 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:08.534435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeda72d-51a0-48ad-8743-ad3808f44102 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:08.534547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeda72d-51a0-48ad-8743-ad3808f44102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:08.553132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-beeda72d-51a0-48ad-8743-ad3808f44102 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:08.554305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-beeda72d-51a0-48ad-8743-ad3808f44102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:08.562458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:08.598203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fbab5-486a-4ee8-8f47-1a0f8c6767bf None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:08.611029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fbab5-486a-4ee8-8f47-1a0f8c6767bf None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:08.615324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28388224-f762-46a7-8fd1-b864581e0726 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:08.615989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388224-f762-46a7-8fd1-b864581e0726 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:08.616270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388224-f762-46a7-8fd1-b864581e0726 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:08.616539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28388224-f762-46a7-8fd1-b864581e0726 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:08.625981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28388224-f762-46a7-8fd1-b864581e0726 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:08.627431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28388224-f762-46a7-8fd1-b864581e0726 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:08.637276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 735/1470] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:18.548166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=17 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:07:18.632583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=17 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:07:41.313506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11b70-4844-4525-a67e-11976ee82b4a None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:41.314131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11b70-4844-4525-a67e-11976ee82b4a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:41.314131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb11b70-4844-4525-a67e-11976ee82b4a None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:41.316198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 736/1471] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:07:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:41.324969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761a97a-60b7-4224-b0f6-e8d17a3c3e77 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:41.325739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761a97a-60b7-4224-b0f6-e8d17a3c3e77 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:41.325908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6761a97a-60b7-4224-b0f6-e8d17a3c3e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:41.349318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6761a97a-60b7-4224-b0f6-e8d17a3c3e77 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:41.350514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6761a97a-60b7-4224-b0f6-e8d17a3c3e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:41.359988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 736/1472] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:41.417653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35704e-8075-4df8-951b-d4d5bf187b77 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:41.417976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35704e-8075-4df8-951b-d4d5bf187b77 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:41.418253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35704e-8075-4df8-951b-d4d5bf187b77 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:41.419666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 737/1473] 10.222.0.69 () {54 vars in 962 bytes} [Fri May 20 09:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:41.424655 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e41d9-d859-4cbf-ae99-9507eb569372 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:41.424941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e41d9-d859-4cbf-ae99-9507eb569372 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:41.425209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e41d9-d859-4cbf-ae99-9507eb569372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:41.443819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-ad4e41d9-d859-4cbf-ae99-9507eb569372 None None] MFA Rules not processed for user `68d25a45f9ee4ae58c6751a8781d12d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:41.445005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ad4e41d9-d859-4cbf-ae99-9507eb569372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:41.451916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 737/1474] 10.222.0.69 () {58 vars in 1059 bytes} [Fri May 20 09:07:41 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:41.462265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35704e-8075-4df8-951b-d4d5bf187b77 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:41.474587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35704e-8075-4df8-951b-d4d5bf187b77 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:41.479083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc799612-8f6a-4c96-b7b7-f57ecec946a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d25a45f9ee4ae58c6751a8781d12d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:41.479858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc799612-8f6a-4c96-b7b7-f57ecec946a7 service glance] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:41.480040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc799612-8f6a-4c96-b7b7-f57ecec946a7 service glance] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:41.480341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc799612-8f6a-4c96-b7b7-f57ecec946a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:41.489515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc799612-8f6a-4c96-b7b7-f57ecec946a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:41.490945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc799612-8f6a-4c96-b7b7-f57ecec946a7 service glance] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:41.500497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 738/1475] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:43.786425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88263-5e86-45bd-bfb5-c65930b04ecd None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:43.786914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88263-5e86-45bd-bfb5-c65930b04ecd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:43.786914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a88263-5e86-45bd-bfb5-c65930b04ecd None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:43.788405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 738/1476] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:43.794080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078b8e-be48-45ad-bd48-bdf1db7a03d3 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:43.794524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078b8e-be48-45ad-bd48-bdf1db7a03d3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:43.794626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078b8e-be48-45ad-bd48-bdf1db7a03d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:43.813342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ea078b8e-be48-45ad-bd48-bdf1db7a03d3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:43.814528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ea078b8e-be48-45ad-bd48-bdf1db7a03d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:43.823768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 739/1477] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:43.857765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a88263-5e86-45bd-bfb5-c65930b04ecd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:43.871862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a88263-5e86-45bd-bfb5-c65930b04ecd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:43.876878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836fbf1-bf0b-495b-bf09-311e0aaab2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:43.877518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836fbf1-bf0b-495b-bf09-311e0aaab2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:43.877795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836fbf1-bf0b-495b-bf09-311e0aaab2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:43.878066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836fbf1-bf0b-495b-bf09-311e0aaab2c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:43.878807 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836fbf1-bf0b-495b-bf09-311e0aaab2c7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:43.880588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING py.warnings [None req-d836fbf1-bf0b-495b-bf09-311e0aaab2c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:07:43.880588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: warnings.warn(msg) May 20 09:07:43.880588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: May 20 09:07:43.881735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836fbf1-bf0b-495b-bf09-311e0aaab2c7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:43.901820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 739/1478] 10.222.0.69 () {56 vars in 1108 bytes} [Fri May 20 09:07:43 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:07:46.102298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee20ee-0b66-47ba-bebe-913ac4ee5f5d None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:46.103146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee20ee-0b66-47ba-bebe-913ac4ee5f5d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:46.103146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee20ee-0b66-47ba-bebe-913ac4ee5f5d None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:46.105363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 740/1479] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:07:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:46.110731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aaee2-f574-448e-b862-533cf2c0afd3 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:46.111180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aaee2-f574-448e-b862-533cf2c0afd3 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:46.111308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503aaee2-f574-448e-b862-533cf2c0afd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:46.130863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-503aaee2-f574-448e-b862-533cf2c0afd3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:46.132115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-503aaee2-f574-448e-b862-533cf2c0afd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:46.141550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 740/1480] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:46.190611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee20ee-0b66-47ba-bebe-913ac4ee5f5d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:46.209291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee20ee-0b66-47ba-bebe-913ac4ee5f5d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:46.215526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731805d1-1ff3-4ab0-8579-7671ef6a5e28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:46.216168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731805d1-1ff3-4ab0-8579-7671ef6a5e28 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:46.216449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731805d1-1ff3-4ab0-8579-7671ef6a5e28 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:46.216721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731805d1-1ff3-4ab0-8579-7671ef6a5e28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:46.225418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731805d1-1ff3-4ab0-8579-7671ef6a5e28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:46.227027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731805d1-1ff3-4ab0-8579-7671ef6a5e28 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:46.241011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 741/1481] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:48.610505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7a1f3-317f-4020-931a-ed26c7cc6a3e None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:48.611335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7a1f3-317f-4020-931a-ed26c7cc6a3e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:48.611335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba7a1f3-317f-4020-931a-ed26c7cc6a3e None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:48.613514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 741/1482] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:07:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:48.620785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8bb2-0332-4fce-b812-e99edc5c9f7a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:48.621254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8bb2-0332-4fce-b812-e99edc5c9f7a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:48.621348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b8bb2-0332-4fce-b812-e99edc5c9f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:48.650458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3f9b8bb2-0332-4fce-b812-e99edc5c9f7a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:48.652244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3f9b8bb2-0332-4fce-b812-e99edc5c9f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:48.662082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 742/1483] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:48.698537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7a1f3-317f-4020-931a-ed26c7cc6a3e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:48.701607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2ba7a1f3-317f-4020-931a-ed26c7cc6a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:48.720026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba7a1f3-317f-4020-931a-ed26c7cc6a3e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:48.724402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699cf2b-728d-410a-93c1-a085916c0bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:48.725047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699cf2b-728d-410a-93c1-a085916c0bfd service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:48.725333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699cf2b-728d-410a-93c1-a085916c0bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:48.725610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699cf2b-728d-410a-93c1-a085916c0bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:48.734886 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699cf2b-728d-410a-93c1-a085916c0bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:48.736365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699cf2b-728d-410a-93c1-a085916c0bfd service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:48.745967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 742/1484] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:50.998997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea414d-dcb5-4751-98cd-562c7157ac0d None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:51.000023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea414d-dcb5-4751-98cd-562c7157ac0d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:51.000023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea414d-dcb5-4751-98cd-562c7157ac0d None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:51.002204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 743/1485] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:07:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:51.008639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad727b92-f582-46cc-8066-0c58685fe8a7 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:51.009325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad727b92-f582-46cc-8066-0c58685fe8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:51.009503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad727b92-f582-46cc-8066-0c58685fe8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:51.034346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-ad727b92-f582-46cc-8066-0c58685fe8a7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:51.035496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ad727b92-f582-46cc-8066-0c58685fe8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:51.044490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 743/1486] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:51.081914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea414d-dcb5-4751-98cd-562c7157ac0d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:51.094438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea414d-dcb5-4751-98cd-562c7157ac0d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:51.098905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd3b74-8f46-4767-98a5-2bba6ea512cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:51.099601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3b74-8f46-4767-98a5-2bba6ea512cc service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:51.099875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3b74-8f46-4767-98a5-2bba6ea512cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:51.100145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd3b74-8f46-4767-98a5-2bba6ea512cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:51.109642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd3b74-8f46-4767-98a5-2bba6ea512cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:51.111073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbd3b74-8f46-4767-98a5-2bba6ea512cc service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:51.121361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 744/1487] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:53.492254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d4e4b-49b6-4b70-bea7-b130b4118b91 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:53.493118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d4e4b-49b6-4b70-bea7-b130b4118b91 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:53.493118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d4e4b-49b6-4b70-bea7-b130b4118b91 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:53.495333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 744/1488] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:07:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:53.504567 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897212c-13ad-4738-b624-0d5cc191173a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:53.505304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897212c-13ad-4738-b624-0d5cc191173a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:53.505481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f897212c-13ad-4738-b624-0d5cc191173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:53.526559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f897212c-13ad-4738-b624-0d5cc191173a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:53.527803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f897212c-13ad-4738-b624-0d5cc191173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:53.537129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 745/1489] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:07:53.577644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d4e4b-49b6-4b70-bea7-b130b4118b91 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:53.589939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d4e4b-49b6-4b70-bea7-b130b4118b91 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:53.594048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c365494-9690-4a0e-b31f-52a30fd4b15a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:53.594683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365494-9690-4a0e-b31f-52a30fd4b15a service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:53.594963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365494-9690-4a0e-b31f-52a30fd4b15a service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:53.595217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c365494-9690-4a0e-b31f-52a30fd4b15a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:53.604236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c365494-9690-4a0e-b31f-52a30fd4b15a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:53.605739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c365494-9690-4a0e-b31f-52a30fd4b15a service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:53.614900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 745/1490] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:07:56.040187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153482-13f4-46e9-b708-3577238adc6e None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:56.040763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153482-13f4-46e9-b708-3577238adc6e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:56.040763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f153482-13f4-46e9-b708-3577238adc6e None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:56.042012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 746/1491] 10.222.0.69 () {54 vars in 926 bytes} [Fri May 20 09:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:07:56.048429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9383-be5e-45c4-b8ea-e6befda57844 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:56.049175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9383-be5e-45c4-b8ea-e6befda57844 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:56.049367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a9383-be5e-45c4-b8ea-e6befda57844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:56.071019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-873a9383-be5e-45c4-b8ea-e6befda57844 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:07:56.072269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-873a9383-be5e-45c4-b8ea-e6befda57844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:07:56.080897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 746/1492] 10.222.0.69 () {58 vars in 1023 bytes} [Fri May 20 09:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:07:56.119382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f153482-13f4-46e9-b708-3577238adc6e None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:07:56.132191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f153482-13f4-46e9-b708-3577238adc6e None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:07:56.136178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c84264a-c8b8-4387-8d6e-e4c1bc89916e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:07:56.136817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84264a-c8b8-4387-8d6e-e4c1bc89916e service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:07:56.137099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84264a-c8b8-4387-8d6e-e4c1bc89916e service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:07:56.137392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c84264a-c8b8-4387-8d6e-e4c1bc89916e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:07:56.147221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84264a-c8b8-4387-8d6e-e4c1bc89916e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:07:56.148686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c84264a-c8b8-4387-8d6e-e4c1bc89916e service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:07:56.158774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 747/1493] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:08:06.065719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=33 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:08:06.154188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=33 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:09:35.005024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459836-a816-4056-a12f-8b7692c31755 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.005657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459836-a816-4056-a12f-8b7692c31755 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.005657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d459836-a816-4056-a12f-8b7692c31755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.027936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-7d459836-a816-4056-a12f-8b7692c31755 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:09:35.029049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7d459836-a816-4056-a12f-8b7692c31755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:09:35.038501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 747/1494] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:09:35.049029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7d5b2-37d7-4d1f-8514-1ebad4e8128c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.049839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7d5b2-37d7-4d1f-8514-1ebad4e8128c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.049839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7d5b2-37d7-4d1f-8514-1ebad4e8128c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.072434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c5b7d5b2-37d7-4d1f-8514-1ebad4e8128c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:09:35.073629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c5b7d5b2-37d7-4d1f-8514-1ebad4e8128c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:09:35.082101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 748/1495] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:09:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:09:35.086735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459836-a816-4056-a12f-8b7692c31755 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:35.096587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d459836-a816-4056-a12f-8b7692c31755 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:35.101499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:35.102147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.102425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.102697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.103470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:09:35.103773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:35.104788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING py.warnings [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 20 09:09:35.104788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: warnings.warn(msg) May 20 09:09:35.104788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: May 20 09:09:35.105705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:35.117187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 748/1496] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:09:35.125226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7d5b2-37d7-4d1f-8514-1ebad4e8128c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:35.134826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b7d5b2-37d7-4d1f-8514-1ebad4e8128c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:35.140623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:35.141276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.141558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.141833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.142529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:35.144347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING py.warnings [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:09:35.144347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: warnings.warn(msg) May 20 09:09:35.144347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: May 20 09:09:35.145549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:35.185776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 749/1497] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:09:35 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:09:35.190058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:35.199947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f614179-4d77-4062-b643-e9e21f2dd1b4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:35.266861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:35.267551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.267833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.268108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.268789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:35.269805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING py.warnings [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:09:35.269805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: warnings.warn(msg) May 20 09:09:35.269805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: May 20 09:09:35.271016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:35.295409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 749/1498] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:09:35 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:09:35.301673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:35.318091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816549c-e55b-48de-9c8d-92b72ea0e30b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:35.326186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:35.326846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.327126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.327427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.328160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:35.329182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING py.warnings [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 20 09:09:35.329182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: warnings.warn(msg) May 20 09:09:35.329182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: May 20 09:09:35.330053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:35.342856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 750/1499] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:09:35.346853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:35.356978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37afae8-56f5-417a-a9e0-46830baa82e3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:35.362188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f21aa96-7ab4-416d-93d4-8dc8e17fb279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:35.362840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21aa96-7ab4-416d-93d4-8dc8e17fb279 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.363124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21aa96-7ab4-416d-93d4-8dc8e17fb279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.363433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f21aa96-7ab4-416d-93d4-8dc8e17fb279 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.364167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21aa96-7ab4-416d-93d4-8dc8e17fb279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:35.365187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING py.warnings [None req-1f21aa96-7ab4-416d-93d4-8dc8e17fb279 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 20 09:09:35.365187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: warnings.warn(msg) May 20 09:09:35.365187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: May 20 09:09:35.366036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f21aa96-7ab4-416d-93d4-8dc8e17fb279 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:35.379434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 750/1500] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:09:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:09:35.383434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:35.394626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052ba4d-b882-4a93-8fd5-c5e66e2d9514 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:35.400146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83dab5b-b267-4a33-aac6-615324b9e0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:35.401043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dab5b-b267-4a33-aac6-615324b9e0b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.401329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dab5b-b267-4a33-aac6-615324b9e0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.401609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83dab5b-b267-4a33-aac6-615324b9e0b6 admin admin] PATH_INFO: `/v3/projects/7da6e8e2e20c4012bdc9901f2ccd04c7/users/f38c8644358b4ac7bb00ee0286ce0bb7/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.416781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dab5b-b267-4a33-aac6-615324b9e0b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7da6e8e2e20c4012bdc9901f2ccd04c7, user_id=f38c8644358b4ac7bb00ee0286ce0bb7, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:35.417804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING py.warnings [None req-f83dab5b-b267-4a33-aac6-615324b9e0b6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:09:35.417804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: warnings.warn(msg) May 20 09:09:35.417804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: May 20 09:09:35.419841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83dab5b-b267-4a33-aac6-615324b9e0b6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:35.435669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 751/1501] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:09:35 2022] PUT /identity/v3/projects/7da6e8e2e20c4012bdc9901f2ccd04c7/users/f38c8644358b4ac7bb00ee0286ce0bb7/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:09:35.447020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5906a-5e5c-463c-a4b1-ea85798a9d35 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.447308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5906a-5e5c-463c-a4b1-ea85798a9d35 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.447617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5906a-5e5c-463c-a4b1-ea85798a9d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.473824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-43b5906a-5e5c-463c-a4b1-ea85798a9d35 None None] MFA Rules not processed for user `f38c8644358b4ac7bb00ee0286ce0bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:09:35.475065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-43b5906a-5e5c-463c-a4b1-ea85798a9d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:09:35.545554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 751/1502] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:09:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:09:35.560102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de635ad-108a-4053-bc5b-196a0d35d012 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.560408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de635ad-108a-4053-bc5b-196a0d35d012 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.560678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de635ad-108a-4053-bc5b-196a0d35d012 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.562052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 752/1503] 10.222.0.69 () {54 vars in 963 bytes} [Fri May 20 09:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:09:35.567047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc020d35-7be6-4bf9-aee3-99657d889e83 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.567372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc020d35-7be6-4bf9-aee3-99657d889e83 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.567659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc020d35-7be6-4bf9-aee3-99657d889e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.714212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-cc020d35-7be6-4bf9-aee3-99657d889e83 None None] MFA Rules not processed for user `4b672a2898b849338844b1c8e911646d`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:09:35.715495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cc020d35-7be6-4bf9-aee3-99657d889e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:09:35.763686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:09:35.773115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 752/1504] 10.222.0.69 () {58 vars in 1060 bytes} [Fri May 20 09:09:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:09:35.783419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de635ad-108a-4053-bc5b-196a0d35d012 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:35.794671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de635ad-108a-4053-bc5b-196a0d35d012 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:35.798376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eee47d-e48d-4354-b158-d5c5f414272a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:35.798956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eee47d-e48d-4354-b158-d5c5f414272a service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.799233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eee47d-e48d-4354-b158-d5c5f414272a service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.799535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eee47d-e48d-4354-b158-d5c5f414272a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.807692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eee47d-e48d-4354-b158-d5c5f414272a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:35.809143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0eee47d-e48d-4354-b158-d5c5f414272a service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:35.818113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 753/1505] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:09:35 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:09:35.860573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e1298-9e1e-4994-9961-494367bc02f6 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.860860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e1298-9e1e-4994-9961-494367bc02f6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.861169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e1298-9e1e-4994-9961-494367bc02f6 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.863169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 753/1506] 10.222.0.69 () {52 vars in 829 bytes} [Fri May 20 09:09:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:09:35.866955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eee47d-e48d-4354-b158-d5c5f414272a service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:35.930226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eee47d-e48d-4354-b158-d5c5f414272a service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:35.935355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:35.936228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:35.936504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:35.936775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] PATH_INFO: `/v3/users/f38c8644358b4ac7bb00ee0286ce0bb7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:35.938037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f38c8644358b4ac7bb00ee0286ce0bb7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:35.939055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING py.warnings [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:09:35.939055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: warnings.warn(msg) May 20 09:09:35.939055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: May 20 09:09:35.940332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:36.008030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f38c8644358b4ac7bb00ee0286ce0bb7'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.008757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f38c8644358b4ac7bb00ee0286ce0bb7'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.009375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f38c8644358b4ac7bb00ee0286ce0bb7'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.022396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f38c8644358b4ac7bb00ee0286ce0bb7'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.035510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f38c8644358b4ac7bb00ee0286ce0bb7'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.051480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:09:36.062381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 754/1507] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:09:35 2022] DELETE /identity/v3/users/f38c8644358b4ac7bb00ee0286ce0bb7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:09:36.067726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ade2-4f5f-4411-9ed5-dd9d5dcdf1dd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:36.068188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ade2-4f5f-4411-9ed5-dd9d5dcdf1dd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:36.068292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa2ade2-4f5f-4411-9ed5-dd9d5dcdf1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:36.088096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6aa2ade2-4f5f-4411-9ed5-dd9d5dcdf1dd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:09:36.089293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6aa2ade2-4f5f-4411-9ed5-dd9d5dcdf1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:09:36.146844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 754/1508] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:09:36.156822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:36.158329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:09:36.217433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbfafd5-48c8-444f-9a00-bfd1f68b1219 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:36.221413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1524feb0-32dc-44f0-ba73-827bf5e162f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:36.222061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524feb0-32dc-44f0-ba73-827bf5e162f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:36.222340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524feb0-32dc-44f0-ba73-827bf5e162f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:36.222612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1524feb0-32dc-44f0-ba73-827bf5e162f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:36.231875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524feb0-32dc-44f0-ba73-827bf5e162f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:36.233355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1524feb0-32dc-44f0-ba73-827bf5e162f4 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:36.242742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 755/1509] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:09:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:09:36.600976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2ade2-4f5f-4411-9ed5-dd9d5dcdf1dd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:09:36.602582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6aa2ade2-4f5f-4411-9ed5-dd9d5dcdf1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:09:36.614680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa2ade2-4f5f-4411-9ed5-dd9d5dcdf1dd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:09:36.621146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:09:36.622149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:09:36.622353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:09:36.622742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] PATH_INFO: `/v3/projects/7da6e8e2e20c4012bdc9901f2ccd04c7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:09:36.624053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7da6e8e2e20c4012bdc9901f2ccd04c7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:09:36.625957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING py.warnings [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:09:36.625957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: warnings.warn(msg) May 20 09:09:36.625957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: May 20 09:09:36.627529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:09:36.691760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7da6e8e2e20c4012bdc9901f2ccd04c7', 'request_id': 'req-a471b452-67b8-42fc-9436-8244eee21f62'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.698491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7da6e8e2e20c4012bdc9901f2ccd04c7', 'request_id': 'req-a471b452-67b8-42fc-9436-8244eee21f62'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.711770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7da6e8e2e20c4012bdc9901f2ccd04c7', 'request_id': 'req-a471b452-67b8-42fc-9436-8244eee21f62'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.729331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] The token cache is being invalidate because project 7da6e8e2e20c4012bdc9901f2ccd04c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:09:36.729608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.730252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a471b452-67b8-42fc-9436-8244eee21f62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:09:36.732033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 755/1510] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:09:36 2022] DELETE /identity/v3/projects/7da6e8e2e20c4012bdc9901f2ccd04c7 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:09:46.238746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:09:46.726254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:11:17.194295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8f586-3331-4e68-b04a-d273d8d628c9 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:17.194911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8f586-3331-4e68-b04a-d273d8d628c9 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:17.194911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e8f586-3331-4e68-b04a-d273d8d628c9 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:17.196281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:11:17.201394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910adf6c-af01-4d17-863b-c858cbebadf1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:17.201863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910adf6c-af01-4d17-863b-c858cbebadf1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:17.201987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910adf6c-af01-4d17-863b-c858cbebadf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:17.221998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-910adf6c-af01-4d17-863b-c858cbebadf1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:17.223213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-910adf6c-af01-4d17-863b-c858cbebadf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:17.283651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:11:17.458590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8f586-3331-4e68-b04a-d273d8d628c9 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:17.460476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-05e8f586-3331-4e68-b04a-d273d8d628c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:17.500463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d97-11d5-4dc9-949f-3abcdbc2d593 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:17.500740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d97-11d5-4dc9-949f-3abcdbc2d593 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:17.501015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5267d97-11d5-4dc9-949f-3abcdbc2d593 None None] PATH_INFO: `/v3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:17.502332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 757/1513] 10.222.0.44 () {54 vars in 925 bytes} [Fri May 20 09:11:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:11:17.529556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002e350-de0b-4474-a86b-4fcfc330883c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:17.529861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002e350-de0b-4474-a86b-4fcfc330883c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:17.530137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002e350-de0b-4474-a86b-4fcfc330883c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:17.545583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e8f586-3331-4e68-b04a-d273d8d628c9 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:17.549953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-7002e350-de0b-4474-a86b-4fcfc330883c None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:17.551153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7002e350-de0b-4474-a86b-4fcfc330883c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:17.552130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e481e-bb87-42df-99b8-85c742948d53 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:17.552940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e481e-bb87-42df-99b8-85c742948d53 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:17.553067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e481e-bb87-42df-99b8-85c742948d53 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:17.553361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e481e-bb87-42df-99b8-85c742948d53 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:17.563338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e481e-bb87-42df-99b8-85c742948d53 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:17.565133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e481e-bb87-42df-99b8-85c742948d53 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:17.575780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 757/1514] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:17.615790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 758/1515] 10.222.0.44 () {58 vars in 1015 bytes} [Fri May 20 09:11:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:11:18.486166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b2f18-9c5e-44b9-aab7-eed220eaf214 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:18.486715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b2f18-9c5e-44b9-aab7-eed220eaf214 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:18.486715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5b2f18-9c5e-44b9-aab7-eed220eaf214 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:18.488280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 758/1516] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:11:18.493123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e61a-87ff-4d62-ab88-916cc36580a0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:18.493559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e61a-87ff-4d62-ab88-916cc36580a0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:18.493669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1488e61a-87ff-4d62-ab88-916cc36580a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:18.512215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1488e61a-87ff-4d62-ab88-916cc36580a0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:18.513403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1488e61a-87ff-4d62-ab88-916cc36580a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:18.521382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 759/1517] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:11:18.674503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b2f18-9c5e-44b9-aab7-eed220eaf214 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:18.695041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5b2f18-9c5e-44b9-aab7-eed220eaf214 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:18.701349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f446d-051b-4e3a-9def-c4d51afee6e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:18.701983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f446d-051b-4e3a-9def-c4d51afee6e6 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:18.702260 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f446d-051b-4e3a-9def-c4d51afee6e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:18.702592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f446d-051b-4e3a-9def-c4d51afee6e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:18.714612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f446d-051b-4e3a-9def-c4d51afee6e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:18.716282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f446d-051b-4e3a-9def-c4d51afee6e6 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:18.731783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 759/1518] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:21.529587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c7d2c-c82d-4b4a-8c52-2931429bdc27 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:21.530269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c7d2c-c82d-4b4a-8c52-2931429bdc27 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:21.530269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c7d2c-c82d-4b4a-8c52-2931429bdc27 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:21.532344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 760/1519] 10.222.0.69 () {54 vars in 960 bytes} [Fri May 20 09:11:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:11:21.537604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39379d2-145a-45b3-9bf9-55475c7e4ef5 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:21.538013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39379d2-145a-45b3-9bf9-55475c7e4ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:21.538162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39379d2-145a-45b3-9bf9-55475c7e4ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:21.688707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a39379d2-145a-45b3-9bf9-55475c7e4ef5 None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:21.689925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a39379d2-145a-45b3-9bf9-55475c7e4ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:21.701038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 760/1520] 10.222.0.69 () {58 vars in 1057 bytes} [Fri May 20 09:11:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:11:21.708940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c7d2c-c82d-4b4a-8c52-2931429bdc27 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:21.724303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8c7d2c-c82d-4b4a-8c52-2931429bdc27 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:21.729674 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42ee52-6fa8-4675-879d-82f0ea08af60 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:21.730301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42ee52-6fa8-4675-879d-82f0ea08af60 service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:21.730585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42ee52-6fa8-4675-879d-82f0ea08af60 service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:21.730844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42ee52-6fa8-4675-879d-82f0ea08af60 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:21.739865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42ee52-6fa8-4675-879d-82f0ea08af60 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:21.741449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42ee52-6fa8-4675-879d-82f0ea08af60 service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:21.751724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 761/1521] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:30.085464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3508d3-be07-48e8-a9d4-48cc5e9b66ae None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:30.086261 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3508d3-be07-48e8-a9d4-48cc5e9b66ae None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:30.086261 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3508d3-be07-48e8-a9d4-48cc5e9b66ae None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:30.088503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:11:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:11:30.094299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e17020-e54c-46d4-bb40-32b558e6c82b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:30.094985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e17020-e54c-46d4-bb40-32b558e6c82b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:30.095081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e17020-e54c-46d4-bb40-32b558e6c82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:30.119478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-66e17020-e54c-46d4-bb40-32b558e6c82b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:30.120690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-66e17020-e54c-46d4-bb40-32b558e6c82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:30.130594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:11:30.284561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3508d3-be07-48e8-a9d4-48cc5e9b66ae None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:30.305344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3508d3-be07-48e8-a9d4-48cc5e9b66ae None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:30.310903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aae259-ae85-491d-bc72-5967040937a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:30.311582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aae259-ae85-491d-bc72-5967040937a2 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:30.311868 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aae259-ae85-491d-bc72-5967040937a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:30.312141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aae259-ae85-491d-bc72-5967040937a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:30.320626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aae259-ae85-491d-bc72-5967040937a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:30.322229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aae259-ae85-491d-bc72-5967040937a2 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:30.331601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 762/1524] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:31.222465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b0e2c-07e9-4e0d-95f7-e19b0c6993a9 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:31.223085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b0e2c-07e9-4e0d-95f7-e19b0c6993a9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:31.223085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1b0e2c-07e9-4e0d-95f7-e19b0c6993a9 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:31.225054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri May 20 09:11:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:11:31.230759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6b495-8229-467a-a555-1f6820fa3fa5 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:31.231194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6b495-8229-467a-a555-1f6820fa3fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:31.231326 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b6b495-8229-467a-a555-1f6820fa3fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:31.257205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-91b6b495-8229-467a-a555-1f6820fa3fa5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:31.258420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-91b6b495-8229-467a-a555-1f6820fa3fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:31.267736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri May 20 09:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:11:31.421414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b0e2c-07e9-4e0d-95f7-e19b0c6993a9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:31.444736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1b0e2c-07e9-4e0d-95f7-e19b0c6993a9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:31.451080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77c9fcf-cbe8-4468-9a3e-926bbb6b63e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:31.451751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c9fcf-cbe8-4468-9a3e-926bbb6b63e9 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:31.452027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c9fcf-cbe8-4468-9a3e-926bbb6b63e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:31.452299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77c9fcf-cbe8-4468-9a3e-926bbb6b63e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:31.461342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c9fcf-cbe8-4468-9a3e-926bbb6b63e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:31.462930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77c9fcf-cbe8-4468-9a3e-926bbb6b63e9 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:31.472940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 764/1527] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:35.628820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d5f77-bd1d-4afd-a77e-2683ce740ef1 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:35.629461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d5f77-bd1d-4afd-a77e-2683ce740ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:35.629461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d5f77-bd1d-4afd-a77e-2683ce740ef1 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:35.630735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:11:35.637049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002b856-7175-42b1-b76b-b264b4c3ca48 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:35.637815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002b856-7175-42b1-b76b-b264b4c3ca48 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:35.637993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0002b856-7175-42b1-b76b-b264b4c3ca48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:35.665827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-0002b856-7175-42b1-b76b-b264b4c3ca48 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:35.667084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0002b856-7175-42b1-b76b-b264b4c3ca48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:35.676301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:11:35.719618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d5f77-bd1d-4afd-a77e-2683ce740ef1 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:35.721686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d21d5f77-bd1d-4afd-a77e-2683ce740ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:35.779329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d5f77-bd1d-4afd-a77e-2683ce740ef1 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:35.783760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2557173-c03d-430f-bca2-be90342fc16d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:35.784486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557173-c03d-430f-bca2-be90342fc16d service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:35.784764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557173-c03d-430f-bca2-be90342fc16d service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:35.785031 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2557173-c03d-430f-bca2-be90342fc16d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:35.795182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2557173-c03d-430f-bca2-be90342fc16d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:35.796656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2557173-c03d-430f-bca2-be90342fc16d service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:35.806463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 765/1530] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:37.974157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493747b5-e7f3-4593-8f94-0ce293c3f995 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:37.974755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493747b5-e7f3-4593-8f94-0ce293c3f995 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:37.974863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493747b5-e7f3-4593-8f94-0ce293c3f995 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:37.976932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:11:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:11:37.984841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f352af-0a9c-42c1-9f0a-b1af05ff54fe None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:37.985354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f352af-0a9c-42c1-9f0a-b1af05ff54fe None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:37.985354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f352af-0a9c-42c1-9f0a-b1af05ff54fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:38.005034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e6f352af-0a9c-42c1-9f0a-b1af05ff54fe None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:38.006208 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e6f352af-0a9c-42c1-9f0a-b1af05ff54fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:38.014767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:11:38.049261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493747b5-e7f3-4593-8f94-0ce293c3f995 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:38.063571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493747b5-e7f3-4593-8f94-0ce293c3f995 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:38.069598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b179801-89b1-4659-92b9-a0139bdb192f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:38.070977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179801-89b1-4659-92b9-a0139bdb192f service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:38.070977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179801-89b1-4659-92b9-a0139bdb192f service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:38.070977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b179801-89b1-4659-92b9-a0139bdb192f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:38.086001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b179801-89b1-4659-92b9-a0139bdb192f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:38.087455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b179801-89b1-4659-92b9-a0139bdb192f service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:38.097618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 767/1533] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:41.738514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c02f0-1ca3-4fdc-b239-f5ae0d6da769 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:41.739124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c02f0-1ca3-4fdc-b239-f5ae0d6da769 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:41.739124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9c02f0-1ca3-4fdc-b239-f5ae0d6da769 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:41.740473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:11:41.746867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d623d34-c63a-44b0-b1f6-369288c69f5a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:41.747349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d623d34-c63a-44b0-b1f6-369288c69f5a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:41.747547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d623d34-c63a-44b0-b1f6-369288c69f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:41.769292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1d623d34-c63a-44b0-b1f6-369288c69f5a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:41.770490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1d623d34-c63a-44b0-b1f6-369288c69f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:41.780650 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:11:42.053295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c02f0-1ca3-4fdc-b239-f5ae0d6da769 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:42.055288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9b9c02f0-1ca3-4fdc-b239-f5ae0d6da769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:42.110987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:11:42.124549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9c02f0-1ca3-4fdc-b239-f5ae0d6da769 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:42.128539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33f193-6912-45b4-8cfb-d82e702a669a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:42.129183 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33f193-6912-45b4-8cfb-d82e702a669a service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:42.129469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33f193-6912-45b4-8cfb-d82e702a669a service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:42.129749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33f193-6912-45b4-8cfb-d82e702a669a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:42.139040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33f193-6912-45b4-8cfb-d82e702a669a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:42.140519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33f193-6912-45b4-8cfb-d82e702a669a service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:42.150166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 768/1536] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:44.505464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34e2b3-fb28-433d-9172-fa71d0f8958c None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:44.506463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34e2b3-fb28-433d-9172-fa71d0f8958c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:44.506463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34e2b3-fb28-433d-9172-fa71d0f8958c None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:44.508728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri May 20 09:11:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:11:44.517506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc3bab-e766-494e-b3cf-961261b39444 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:44.518189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc3bab-e766-494e-b3cf-961261b39444 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:44.518440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc3bab-e766-494e-b3cf-961261b39444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:44.538392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6bbc3bab-e766-494e-b3cf-961261b39444 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:11:44.539639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6bbc3bab-e766-494e-b3cf-961261b39444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:44.548386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri May 20 09:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:11:44.836433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34e2b3-fb28-433d-9172-fa71d0f8958c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:11:44.839912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0a34e2b3-fb28-433d-9172-fa71d0f8958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:11:44.856847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34e2b3-fb28-433d-9172-fa71d0f8958c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:11:44.860767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d385c918-a991-4d08-ac34-1bb263e4732d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:11:44.861431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385c918-a991-4d08-ac34-1bb263e4732d service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:11:44.861709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385c918-a991-4d08-ac34-1bb263e4732d service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:11:44.861979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d385c918-a991-4d08-ac34-1bb263e4732d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:11:44.871799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385c918-a991-4d08-ac34-1bb263e4732d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:11:44.873348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d385c918-a991-4d08-ac34-1bb263e4732d service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:11:44.883411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 770/1539] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:11:54.533626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=10 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:11:54.878911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:13.369690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85995-0174-42c8-baad-03af6b44d8d2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.370844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85995-0174-42c8-baad-03af6b44d8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.371389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f85995-0174-42c8-baad-03af6b44d8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.406314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-69f85995-0174-42c8-baad-03af6b44d8d2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.407926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-69f85995-0174-42c8-baad-03af6b44d8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.422323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 770/1540] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.433733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21c6d6-7e76-4f14-b2ab-3f3e4eff662a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.434358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21c6d6-7e76-4f14-b2ab-3f3e4eff662a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.434358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21c6d6-7e76-4f14-b2ab-3f3e4eff662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.467310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-9f21c6d6-7e76-4f14-b2ab-3f3e4eff662a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.468755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9f21c6d6-7e76-4f14-b2ab-3f3e4eff662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.479577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 771/1541] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.491582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85995-0174-42c8-baad-03af6b44d8d2 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:13.512001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f85995-0174-42c8-baad-03af6b44d8d2 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:13.519889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756092b1-5cac-47d7-8209-5e33bd9dac29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:13.521041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756092b1-5cac-47d7-8209-5e33bd9dac29 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.521540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756092b1-5cac-47d7-8209-5e33bd9dac29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.522041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756092b1-5cac-47d7-8209-5e33bd9dac29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.523225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756092b1-5cac-47d7-8209-5e33bd9dac29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:13.523805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756092b1-5cac-47d7-8209-5e33bd9dac29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:13.525094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING py.warnings [None req-756092b1-5cac-47d7-8209-5e33bd9dac29 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 20 09:13:13.525094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: warnings.warn(msg) May 20 09:13:13.525094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: May 20 09:13:13.526762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756092b1-5cac-47d7-8209-5e33bd9dac29 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:13.543775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 771/1542] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:13.552364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21c6d6-7e76-4f14-b2ab-3f3e4eff662a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:13.565470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21c6d6-7e76-4f14-b2ab-3f3e4eff662a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:13.574551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560f636-f845-43f4-a2e1-ceac2bce5303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:13.575668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560f636-f845-43f4-a2e1-ceac2bce5303 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.576178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560f636-f845-43f4-a2e1-ceac2bce5303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.576667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560f636-f845-43f4-a2e1-ceac2bce5303 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.578259 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560f636-f845-43f4-a2e1-ceac2bce5303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:13.579885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING py.warnings [None req-4560f636-f845-43f4-a2e1-ceac2bce5303 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:13:13.579885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: warnings.warn(msg) May 20 09:13:13.579885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: May 20 09:13:13.581493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560f636-f845-43f4-a2e1-ceac2bce5303 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:13.589762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cda24f-38d1-450b-9980-e036393ad8e6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.590752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cda24f-38d1-450b-9980-e036393ad8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.591340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cda24f-38d1-450b-9980-e036393ad8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.621216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c1cda24f-38d1-450b-9980-e036393ad8e6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.623176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c1cda24f-38d1-450b-9980-e036393ad8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.638426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 772/1543] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.642304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 772/1544] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/projects => generated 391 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:13.646546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cda24f-38d1-450b-9980-e036393ad8e6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:13.653045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3ec07-fd63-4de4-8272-736cae586d65 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.653759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3ec07-fd63-4de4-8272-736cae586d65 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.654254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf3ec07-fd63-4de4-8272-736cae586d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.664747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cda24f-38d1-450b-9980-e036393ad8e6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:13.683651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-cdf3ec07-fd63-4de4-8272-736cae586d65 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.685341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cdf3ec07-fd63-4de4-8272-736cae586d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.756592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896113a4-ffa1-4417-8845-982952a805d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:13.757290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896113a4-ffa1-4417-8845-982952a805d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.757570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896113a4-ffa1-4417-8845-982952a805d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.757841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896113a4-ffa1-4417-8845-982952a805d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.758761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896113a4-ffa1-4417-8845-982952a805d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:13.759831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING py.warnings [None req-896113a4-ffa1-4417-8845-982952a805d4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:13:13.759831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: warnings.warn(msg) May 20 09:13:13.759831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: May 20 09:13:13.761024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896113a4-ffa1-4417-8845-982952a805d4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:13.784146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 773/1545] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:13.785905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c945d-6e06-4cc3-aed8-f7a184892778 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.786194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c945d-6e06-4cc3-aed8-f7a184892778 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.786592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c945d-6e06-4cc3-aed8-f7a184892778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.792068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 773/1546] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/users => generated 296 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:13.793065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffae5c-89cc-4884-b78b-814b9a936db5 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.793901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffae5c-89cc-4884-b78b-814b9a936db5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.793901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffae5c-89cc-4884-b78b-814b9a936db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.809418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-074c945d-6e06-4cc3-aed8-f7a184892778 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.811093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-074c945d-6e06-4cc3-aed8-f7a184892778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.812819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f0ffae5c-89cc-4884-b78b-814b9a936db5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.813966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f0ffae5c-89cc-4884-b78b-814b9a936db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.820450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 774/1547] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.822163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe9a-4192-49d1-b90c-7969cb21bd65 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.822632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe9a-4192-49d1-b90c-7969cb21bd65 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.823071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecdfe9a-4192-49d1-b90c-7969cb21bd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.830194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 774/1548] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.831536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdee64-b545-4858-9b88-59146f91d4f6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.831820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdee64-b545-4858-9b88-59146f91d4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.832089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cdee64-b545-4858-9b88-59146f91d4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.842534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-cecdfe9a-4192-49d1-b90c-7969cb21bd65 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.843764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cecdfe9a-4192-49d1-b90c-7969cb21bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.850021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-44cdee64-b545-4858-9b88-59146f91d4f6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.851163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-44cdee64-b545-4858-9b88-59146f91d4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.851624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 775/1549] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.853137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383f354-17a0-4400-9627-556bc4cd31cd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.853425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383f354-17a0-4400-9627-556bc4cd31cd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.853701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c383f354-17a0-4400-9627-556bc4cd31cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.862280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 775/1550] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.863112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdee64-b545-4858-9b88-59146f91d4f6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:13.872044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-c383f354-17a0-4400-9627-556bc4cd31cd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.873162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cdee64-b545-4858-9b88-59146f91d4f6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:13.873287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c383f354-17a0-4400-9627-556bc4cd31cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.877760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d9200a-1004-4aac-bf5a-4dd39748639b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:13.878416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9200a-1004-4aac-bf5a-4dd39748639b admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.878733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9200a-1004-4aac-bf5a-4dd39748639b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.879009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d9200a-1004-4aac-bf5a-4dd39748639b admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.879811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9200a-1004-4aac-bf5a-4dd39748639b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:13.880123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9200a-1004-4aac-bf5a-4dd39748639b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:13.881199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 776/1551] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.889992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d9200a-1004-4aac-bf5a-4dd39748639b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:13.890133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383f354-17a0-4400-9627-556bc4cd31cd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:13.891738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c383f354-17a0-4400-9627-556bc4cd31cd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:13.894396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 776/1552] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:13.895751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a091f-de62-4665-91de-08f347ea0887 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.896043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a091f-de62-4665-91de-08f347ea0887 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.896311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a091f-de62-4665-91de-08f347ea0887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.896599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bc1566-4787-4d14-9d3e-9e372aa5dcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:13.897276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc1566-4787-4d14-9d3e-9e372aa5dcf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.897597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc1566-4787-4d14-9d3e-9e372aa5dcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.897826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bc1566-4787-4d14-9d3e-9e372aa5dcf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.898691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc1566-4787-4d14-9d3e-9e372aa5dcf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:13.900295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bc1566-4787-4d14-9d3e-9e372aa5dcf4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:13.913800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 777/1553] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:13.913800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87095b2-9783-4deb-8599-724788fc7945 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.913800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87095b2-9783-4deb-8599-724788fc7945 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.913800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87095b2-9783-4deb-8599-724788fc7945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.914257 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-8a5a091f-de62-4665-91de-08f347ea0887 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.915433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8a5a091f-de62-4665-91de-08f347ea0887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.924324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 777/1554] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.926055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd673-0c77-4aab-bd04-c78e16de4b94 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.926493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd673-0c77-4aab-bd04-c78e16de4b94 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.926908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cd673-0c77-4aab-bd04-c78e16de4b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.930387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e87095b2-9783-4deb-8599-724788fc7945 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.931599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e87095b2-9783-4deb-8599-724788fc7945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.939559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 778/1555] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.940656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201981cc-29f2-44dc-9035-2934ba8ba4b3 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.940932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201981cc-29f2-44dc-9035-2934ba8ba4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.941204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201981cc-29f2-44dc-9035-2934ba8ba4b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.949187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-731cd673-0c77-4aab-bd04-c78e16de4b94 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.950368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-731cd673-0c77-4aab-bd04-c78e16de4b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.958344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 778/1556] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.958548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-201981cc-29f2-44dc-9035-2934ba8ba4b3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.959655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901546d1-e9d9-4779-b0ed-e1f47e5689b0 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.959789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-201981cc-29f2-44dc-9035-2934ba8ba4b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.959966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901546d1-e9d9-4779-b0ed-e1f47e5689b0 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.960211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901546d1-e9d9-4779-b0ed-e1f47e5689b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.967456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 779/1557] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.968650 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ed693-5040-4863-becd-a19f0a3ab7ec None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:13.968935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ed693-5040-4863-becd-a19f0a3ab7ec None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:13.969240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ed693-5040-4863-becd-a19f0a3ab7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:13.978420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-901546d1-e9d9-4779-b0ed-e1f47e5689b0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.979665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-901546d1-e9d9-4779-b0ed-e1f47e5689b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.985987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-723ed693-5040-4863-becd-a19f0a3ab7ec None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:13.987103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-723ed693-5040-4863-becd-a19f0a3ab7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:13.988680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 779/1558] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.989467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901546d1-e9d9-4779-b0ed-e1f47e5689b0 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:13.996172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 780/1559] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:13.997022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ed693-5040-4863-becd-a19f0a3ab7ec None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:13.998888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901546d1-e9d9-4779-b0ed-e1f47e5689b0 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.003923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d27955-33e9-487b-891b-a68da6b7d0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.004574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27955-33e9-487b-891b-a68da6b7d0f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.004856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27955-33e9-487b-891b-a68da6b7d0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.005129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d27955-33e9-487b-891b-a68da6b7d0f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.005819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d27955-33e9-487b-891b-a68da6b7d0f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.006216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ed693-5040-4863-becd-a19f0a3ab7ec None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.007792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d27955-33e9-487b-891b-a68da6b7d0f4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.010768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a07d6-cde7-4666-a678-a9c59ea75b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.011429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a07d6-cde7-4666-a678-a9c59ea75b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.011692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a07d6-cde7-4666-a678-a9c59ea75b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.011969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a07d6-cde7-4666-a678-a9c59ea75b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.012687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a07d6-cde7-4666-a678-a9c59ea75b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.014321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a07d6-cde7-4666-a678-a9c59ea75b91 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.023963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 781/1560] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:14.025729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f4ac-e351-4891-b290-3b54fd134e19 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.026055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f4ac-e351-4891-b290-3b54fd134e19 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.026377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211f4ac-e351-4891-b290-3b54fd134e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.039936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 780/1561] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:13 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.039936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d27955-33e9-487b-891b-a68da6b7d0f4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.044966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a211f4ac-e351-4891-b290-3b54fd134e19 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:14.046157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a211f4ac-e351-4891-b290-3b54fd134e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:14.096510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:14.104605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d27955-33e9-487b-891b-a68da6b7d0f4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.106050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 782/1562] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:14.107018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211f4ac-e351-4891-b290-3b54fd134e19 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.110601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.111615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.112027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.112306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.113643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:14.113959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.115762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.116137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211f4ac-e351-4891-b290-3b54fd134e19 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.121018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.121906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.122146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.122380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.123162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:14.123691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.125322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.134065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 781/1563] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:14.135118 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.136125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:14.146948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 783/1564] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:14.148157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.150867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a862e22d-4b96-465f-b0ac-4638ae1fc6be admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.155952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.156662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.156913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.157192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.157927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:14.158234 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.158395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ba2f0-27e9-42cb-81a2-593c6a48d0e7 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.160131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.163564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.164255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.164546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.164828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.165588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:14.165898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.168003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.172641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 782/1565] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:14.173310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.182521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b978-458c-44a6-a531-a5d32235673e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.187085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.187748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.187937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 784/1566] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:14.188290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.188584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.188728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.189290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:14.189593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.191184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.198249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a257cc-a318-4249-bacf-ef52b62a7e1f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.204489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.212245 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 783/1567] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:14.212416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.212416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.212416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] PATH_INFO: `/v3/projects/ed0e52a9d6374123b57759435d43d7aa/users/992571802b7d426f923e0f0bc41e28de/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.212767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.217862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3258eb1e-7154-41fa-b55b-00e064cfa730 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.220939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0e52a9d6374123b57759435d43d7aa, user_id=992571802b7d426f923e0f0bc41e28de, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.222025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING py.warnings [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:13:14.222025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: warnings.warn(msg) May 20 09:13:14.222025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: May 20 09:13:14.222353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a53e9-3a81-4ef4-ab6d-c023d25b2cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.223037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a53e9-3a81-4ef4-ab6d-c023d25b2cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.223347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a53e9-3a81-4ef4-ab6d-c023d25b2cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.223622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a53e9-3a81-4ef4-ab6d-c023d25b2cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.223968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.224312 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a53e9-3a81-4ef4-ab6d-c023d25b2cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.226208 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a53e9-3a81-4ef4-ab6d-c023d25b2cd5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.236958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 785/1568] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:14 2022] PUT /identity/v3/projects/ed0e52a9d6374123b57759435d43d7aa/users/992571802b7d426f923e0f0bc41e28de/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:14.237707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.251394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 784/1569] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.252192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a53e9-3a81-4ef4-ab6d-c023d25b2cd5 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.261472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a53e9-3a81-4ef4-ab6d-c023d25b2cd5 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.299608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca23cfc-ee56-4f0e-a843-b298cae046eb admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.304181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.304835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.305120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.305396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.306155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:14.306461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.308112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.315691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0274e-7328-4a7f-80d4-387002401c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.316359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0274e-7328-4a7f-80d4-387002401c85 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.316634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0274e-7328-4a7f-80d4-387002401c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.316928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad0274e-7328-4a7f-80d4-387002401c85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.317720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0274e-7328-4a7f-80d4-387002401c85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.319278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 786/1570] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:14.319671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad0274e-7328-4a7f-80d4-387002401c85 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.320049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.331696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a0b58-2105-4e22-97b7-5adfb6ee61c8 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.336830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fc40a-fb7c-4dcc-8b5f-b0f5ce8cdf74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.337543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fc40a-fb7c-4dcc-8b5f-b0f5ce8cdf74 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.337869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fc40a-fb7c-4dcc-8b5f-b0f5ce8cdf74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.338197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330fc40a-fb7c-4dcc-8b5f-b0f5ce8cdf74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.338868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fc40a-fb7c-4dcc-8b5f-b0f5ce8cdf74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.340946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330fc40a-fb7c-4dcc-8b5f-b0f5ce8cdf74 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.346530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 785/1571] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.347347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0274e-7328-4a7f-80d4-387002401c85 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.368575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 787/1572] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.369518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fc40a-fb7c-4dcc-8b5f-b0f5ce8cdf74 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.413117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad0274e-7328-4a7f-80d4-387002401c85 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.432474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330fc40a-fb7c-4dcc-8b5f-b0f5ce8cdf74 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.437889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ed8f3-062c-4053-ba9c-3ee1ec5185ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.438601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ed8f3-062c-4053-ba9c-3ee1ec5185ba admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.438879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ed8f3-062c-4053-ba9c-3ee1ec5185ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.439150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6ed8f3-062c-4053-ba9c-3ee1ec5185ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.439871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ed8f3-062c-4053-ba9c-3ee1ec5185ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.441806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6ed8f3-062c-4053-ba9c-3ee1ec5185ba admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.445186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:14.468443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 788/1573] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.469401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ed8f3-062c-4053-ba9c-3ee1ec5185ba admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.469675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b939a962-7cf2-4898-add4-5c2f7af7b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.470452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939a962-7cf2-4898-add4-5c2f7af7b8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.470709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939a962-7cf2-4898-add4-5c2f7af7b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.470987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b939a962-7cf2-4898-add4-5c2f7af7b8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.471715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939a962-7cf2-4898-add4-5c2f7af7b8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.473627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b939a962-7cf2-4898-add4-5c2f7af7b8bf admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.492009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:14.504319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 786/1574] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/projects => generated 403 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.504319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba783736-7f95-4efc-8f4b-0962760682eb None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.504319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba783736-7f95-4efc-8f4b-0962760682eb None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.504319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba783736-7f95-4efc-8f4b-0962760682eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.519837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ba783736-7f95-4efc-8f4b-0962760682eb None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:14.521028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ba783736-7f95-4efc-8f4b-0962760682eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:14.536694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6ed8f3-062c-4053-ba9c-3ee1ec5185ba admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.585429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 787/1575] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:14.586296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba783736-7f95-4efc-8f4b-0962760682eb None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.596214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba783736-7f95-4efc-8f4b-0962760682eb None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.601457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2898dee-1e8b-42b6-89b1-13996e40a843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.602121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2898dee-1e8b-42b6-89b1-13996e40a843 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.602398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2898dee-1e8b-42b6-89b1-13996e40a843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.602697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2898dee-1e8b-42b6-89b1-13996e40a843 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.603452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2898dee-1e8b-42b6-89b1-13996e40a843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.604027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3514059-bf27-4b03-92af-a5b91820230d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.604693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3514059-bf27-4b03-92af-a5b91820230d admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.604974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3514059-bf27-4b03-92af-a5b91820230d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.605085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2898dee-1e8b-42b6-89b1-13996e40a843 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.605250 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3514059-bf27-4b03-92af-a5b91820230d admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.605953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3514059-bf27-4b03-92af-a5b91820230d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.607963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3514059-bf27-4b03-92af-a5b91820230d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.615401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 788/1576] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:14.617040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2898dee-1e8b-42b6-89b1-13996e40a843 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.627767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2898dee-1e8b-42b6-89b1-13996e40a843 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.633114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820196a-7426-411f-b11e-c80e45c2ff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.633764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820196a-7426-411f-b11e-c80e45c2ff59 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.634078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820196a-7426-411f-b11e-c80e45c2ff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.634354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0820196a-7426-411f-b11e-c80e45c2ff59 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.634923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 789/1577] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/projects => generated 387 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.635071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820196a-7426-411f-b11e-c80e45c2ff59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.637075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0820196a-7426-411f-b11e-c80e45c2ff59 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.638834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3514059-bf27-4b03-92af-a5b91820230d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.665704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 789/1578] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.666424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820196a-7426-411f-b11e-c80e45c2ff59 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.701923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3514059-bf27-4b03-92af-a5b91820230d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.726941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0820196a-7426-411f-b11e-c80e45c2ff59 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.731787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592486f-8f7f-411b-a708-e338def6d6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.732694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592486f-8f7f-411b-a708-e338def6d6c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.732974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592486f-8f7f-411b-a708-e338def6d6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.733246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592486f-8f7f-411b-a708-e338def6d6c0 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.733941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592486f-8f7f-411b-a708-e338def6d6c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.735933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592486f-8f7f-411b-a708-e338def6d6c0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.752648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:14.756800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 790/1579] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.757707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592486f-8f7f-411b-a708-e338def6d6c0 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.759627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370fdf4-8aad-441d-afb3-97748568fe70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.760471 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370fdf4-8aad-441d-afb3-97748568fe70 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.760750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370fdf4-8aad-441d-afb3-97748568fe70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.761023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370fdf4-8aad-441d-afb3-97748568fe70 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.761714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370fdf4-8aad-441d-afb3-97748568fe70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.763673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370fdf4-8aad-441d-afb3-97748568fe70 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.773913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592486f-8f7f-411b-a708-e338def6d6c0 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.780073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.780722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.781012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.781274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.781960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.783972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.785671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 790/1580] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.786662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370fdf4-8aad-441d-afb3-97748568fe70 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.796111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370fdf4-8aad-441d-afb3-97748568fe70 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.801271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd12e-d845-46bb-8d5a-ddf948fe4f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.801925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd12e-d845-46bb-8d5a-ddf948fe4f84 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.802243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd12e-d845-46bb-8d5a-ddf948fe4f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.802547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355dd12e-d845-46bb-8d5a-ddf948fe4f84 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.803282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd12e-d845-46bb-8d5a-ddf948fe4f84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.805209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355dd12e-d845-46bb-8d5a-ddf948fe4f84 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.807485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 791/1581] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/users => generated 292 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.808305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.811824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:14.818191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:14.831834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 791/1582] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/users => generated 302 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.831834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd12e-d845-46bb-8d5a-ddf948fe4f84 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.839569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355dd12e-d845-46bb-8d5a-ddf948fe4f84 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.845547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66348e8-9268-4fc1-bfac-1ada70cc6fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.846469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66348e8-9268-4fc1-bfac-1ada70cc6fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.848194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66348e8-9268-4fc1-bfac-1ada70cc6fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.848194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66348e8-9268-4fc1-bfac-1ada70cc6fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.849088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66348e8-9268-4fc1-bfac-1ada70cc6fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.852078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66348e8-9268-4fc1-bfac-1ada70cc6fc7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.866349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 792/1583] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:14.867151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66348e8-9268-4fc1-bfac-1ada70cc6fc7 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.876991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66348e8-9268-4fc1-bfac-1ada70cc6fc7 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.881476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08f4c8-855e-4806-a458-11e587fd58a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.882112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08f4c8-855e-4806-a458-11e587fd58a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.882328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74936736-3852-4ef5-a345-4b51da5a5c49 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.882442 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08f4c8-855e-4806-a458-11e587fd58a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.882649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de08f4c8-855e-4806-a458-11e587fd58a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.883379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08f4c8-855e-4806-a458-11e587fd58a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.885273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de08f4c8-855e-4806-a458-11e587fd58a1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.886280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d4edb-fcc7-4e2b-93ae-0831e86fccc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.886938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d4edb-fcc7-4e2b-93ae-0831e86fccc1 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.887213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d4edb-fcc7-4e2b-93ae-0831e86fccc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.887518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62d4edb-fcc7-4e2b-93ae-0831e86fccc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.896775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d4edb-fcc7-4e2b-93ae-0831e86fccc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.898300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62d4edb-fcc7-4e2b-93ae-0831e86fccc1 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.906393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 793/1584] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/users => generated 294 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.908144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08f4c8-855e-4806-a458-11e587fd58a1 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.908931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 792/1585] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:14.909891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d4edb-fcc7-4e2b-93ae-0831e86fccc1 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.918226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de08f4c8-855e-4806-a458-11e587fd58a1 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.919981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62d4edb-fcc7-4e2b-93ae-0831e86fccc1 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.923751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f9a4-e581-4694-ba49-329c1366530a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.924415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f9a4-e581-4694-ba49-329c1366530a admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.924669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f9a4-e581-4694-ba49-329c1366530a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.925014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6f9a4-e581-4694-ba49-329c1366530a admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.925713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f9a4-e581-4694-ba49-329c1366530a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.927588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fffd-8ad2-4827-8cc8-ddd4a37c72e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.927758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb6f9a4-e581-4694-ba49-329c1366530a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.928451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fffd-8ad2-4827-8cc8-ddd4a37c72e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.928925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fffd-8ad2-4827-8cc8-ddd4a37c72e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.929361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f3fffd-8ad2-4827-8cc8-ddd4a37c72e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.930141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3fffd-8ad2-4827-8cc8-ddd4a37c72e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.932129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f3fffd-8ad2-4827-8cc8-ddd4a37c72e2 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.944400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 793/1586] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:14.945153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fffd-8ad2-4827-8cc8-ddd4a37c72e2 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.952061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 794/1587] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:14 2022] POST /identity/v3/users => generated 290 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:14.952061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f9a4-e581-4694-ba49-329c1366530a admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.958147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f3fffd-8ad2-4827-8cc8-ddd4a37c72e2 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.960822 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6f9a4-e581-4694-ba49-329c1366530a admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:14.963818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cbc8d-57ff-4b60-81cd-af5be234f68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.964707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cbc8d-57ff-4b60-81cd-af5be234f68a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.965140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cbc8d-57ff-4b60-81cd-af5be234f68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.965592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8cbc8d-57ff-4b60-81cd-af5be234f68a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.966533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153e458-706e-4c28-92bf-e98628f9ad51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:14.967123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cbc8d-57ff-4b60-81cd-af5be234f68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.967295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153e458-706e-4c28-92bf-e98628f9ad51 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:14.967497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153e458-706e-4c28-92bf-e98628f9ad51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:14.967907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f153e458-706e-4c28-92bf-e98628f9ad51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:14.968644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153e458-706e-4c28-92bf-e98628f9ad51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:14.968930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8cbc8d-57ff-4b60-81cd-af5be234f68a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.970214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f153e458-706e-4c28-92bf-e98628f9ad51 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:14.986405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 794/1588] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:14.987441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 795/1589] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:14.987633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cbc8d-57ff-4b60-81cd-af5be234f68a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:14.987715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153e458-706e-4c28-92bf-e98628f9ad51 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.002782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8cbc8d-57ff-4b60-81cd-af5be234f68a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.003161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f153e458-706e-4c28-92bf-e98628f9ad51 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.008048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdbcb7-2627-4913-a927-f58de18f2bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.008636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2b9ba-16de-42e7-8479-f47fe041a60e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.008830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdbcb7-2627-4913-a927-f58de18f2bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.009211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdbcb7-2627-4913-a927-f58de18f2bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.009517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2b9ba-16de-42e7-8479-f47fe041a60e admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.009634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfdbcb7-2627-4913-a927-f58de18f2bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.009794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2b9ba-16de-42e7-8479-f47fe041a60e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.010074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2b9ba-16de-42e7-8479-f47fe041a60e admin admin] PATH_INFO: `/v3/projects/69203fd149f44cc7830ca471e186ac78/users/de4e89c56a0542a9bb92e7da79a5ead4/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.010627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdbcb7-2627-4913-a927-f58de18f2bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.012394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfdbcb7-2627-4913-a927-f58de18f2bb9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.023650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 795/1590] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.024754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdbcb7-2627-4913-a927-f58de18f2bb9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.030530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2b9ba-16de-42e7-8479-f47fe041a60e admin admin] RBAC: Authorizing `identity:create_grant(project_id=69203fd149f44cc7830ca471e186ac78, user_id=de4e89c56a0542a9bb92e7da79a5ead4, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.033209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2b9ba-16de-42e7-8479-f47fe041a60e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.036297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfdbcb7-2627-4913-a927-f58de18f2bb9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.041868 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a2872-050b-4eae-91a0-584b5c451b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.042606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a2872-050b-4eae-91a0-584b5c451b66 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.042947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a2872-050b-4eae-91a0-584b5c451b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.043379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a2872-050b-4eae-91a0-584b5c451b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.044227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a2872-050b-4eae-91a0-584b5c451b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.045969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a2872-050b-4eae-91a0-584b5c451b66 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.049314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 796/1591] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:14 2022] PUT /identity/v3/projects/69203fd149f44cc7830ca471e186ac78/users/de4e89c56a0542a9bb92e7da79a5ead4/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:15.050144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2b9ba-16de-42e7-8479-f47fe041a60e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.057308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 796/1592] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.058228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a2872-050b-4eae-91a0-584b5c451b66 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.069236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a2872-050b-4eae-91a0-584b5c451b66 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.130680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2b9ba-16de-42e7-8479-f47fe041a60e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.136342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836c6ed-ac8a-480d-8172-f1628462620f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.136541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52bdcc-b19e-43ad-b2be-fcfba9f73cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.137132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52bdcc-b19e-43ad-b2be-fcfba9f73cab admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.137358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52bdcc-b19e-43ad-b2be-fcfba9f73cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.137733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52bdcc-b19e-43ad-b2be-fcfba9f73cab admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.138259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836c6ed-ac8a-480d-8172-f1628462620f admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.138903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52bdcc-b19e-43ad-b2be-fcfba9f73cab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.139308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836c6ed-ac8a-480d-8172-f1628462620f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.140372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4836c6ed-ac8a-480d-8172-f1628462620f admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.141384 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52bdcc-b19e-43ad-b2be-fcfba9f73cab admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.142816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836c6ed-ac8a-480d-8172-f1628462620f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.145697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4836c6ed-ac8a-480d-8172-f1628462620f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.154959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 797/1593] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.155861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52bdcc-b19e-43ad-b2be-fcfba9f73cab admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.158306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 797/1594] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.159384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836c6ed-ac8a-480d-8172-f1628462620f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.167313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52bdcc-b19e-43ad-b2be-fcfba9f73cab admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.171293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4836c6ed-ac8a-480d-8172-f1628462620f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.173054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65d387-4f72-412c-9c61-b387be1d9c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.173892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65d387-4f72-412c-9c61-b387be1d9c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.174302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65d387-4f72-412c-9c61-b387be1d9c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.174663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65d387-4f72-412c-9c61-b387be1d9c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.175524 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65d387-4f72-412c-9c61-b387be1d9c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.176638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e4663-9322-49d2-b2c5-567164426216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.177246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b65d387-4f72-412c-9c61-b387be1d9c8e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.177389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e4663-9322-49d2-b2c5-567164426216 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.177761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e4663-9322-49d2-b2c5-567164426216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.178097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e4663-9322-49d2-b2c5-567164426216 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.178898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e4663-9322-49d2-b2c5-567164426216 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.180674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e4663-9322-49d2-b2c5-567164426216 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.189002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 798/1595] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.189864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65d387-4f72-412c-9c61-b387be1d9c8e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.194865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 798/1596] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.196573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421ea8-dad9-4e56-852e-829971ab9c8d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.197089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421ea8-dad9-4e56-852e-829971ab9c8d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.197369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c421ea8-dad9-4e56-852e-829971ab9c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.202804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65d387-4f72-412c-9c61-b387be1d9c8e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.208350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2820d3-6efd-40a0-9f8b-b20087620e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.209034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2820d3-6efd-40a0-9f8b-b20087620e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.209317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2820d3-6efd-40a0-9f8b-b20087620e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.209592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2820d3-6efd-40a0-9f8b-b20087620e1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.210326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2820d3-6efd-40a0-9f8b-b20087620e1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.211985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2820d3-6efd-40a0-9f8b-b20087620e1d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.227092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 799/1597] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.227844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0c421ea8-dad9-4e56-852e-829971ab9c8d None None] MFA Rules not processed for user `de4e89c56a0542a9bb92e7da79a5ead4`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:15.227982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2820d3-6efd-40a0-9f8b-b20087620e1d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.229818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0c421ea8-dad9-4e56-852e-829971ab9c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:15.241632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2820d3-6efd-40a0-9f8b-b20087620e1d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.247251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5577d-8ab6-438f-9b81-740583dec75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.247993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5577d-8ab6-438f-9b81-740583dec75f admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.248294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5577d-8ab6-438f-9b81-740583dec75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.249210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5577d-8ab6-438f-9b81-740583dec75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.249959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5577d-8ab6-438f-9b81-740583dec75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.252032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5577d-8ab6-438f-9b81-740583dec75f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.267520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 800/1598] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.268570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5577d-8ab6-438f-9b81-740583dec75f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.279591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5577d-8ab6-438f-9b81-740583dec75f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.280148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:15.286359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5e4dd-5a26-4c53-b836-5765304b75f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.287748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5e4dd-5a26-4c53-b836-5765304b75f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.288170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5e4dd-5a26-4c53-b836-5765304b75f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.288535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5e4dd-5a26-4c53-b836-5765304b75f3 admin admin] PATH_INFO: `/v3/projects/9dd35ce1fcbd4038a0c556a45059c7d7/users/bfaaa367491745708334c5f35331bf76/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.300401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:15.309022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5e4dd-5a26-4c53-b836-5765304b75f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dd35ce1fcbd4038a0c556a45059c7d7, user_id=bfaaa367491745708334c5f35331bf76, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.313128 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5e4dd-5a26-4c53-b836-5765304b75f3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.330332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 799/1599] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:15.330899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421ea8-dad9-4e56-852e-829971ab9c8d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.341779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 801/1600] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:15 2022] PUT /identity/v3/projects/9dd35ce1fcbd4038a0c556a45059c7d7/users/bfaaa367491745708334c5f35331bf76/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:15.341779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5e4dd-5a26-4c53-b836-5765304b75f3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.344028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c421ea8-dad9-4e56-852e-829971ab9c8d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.408335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613995f-4f77-4816-b3b0-bd90379b9548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.408999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613995f-4f77-4816-b3b0-bd90379b9548 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.409283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613995f-4f77-4816-b3b0-bd90379b9548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.409554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613995f-4f77-4816-b3b0-bd90379b9548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.410294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613995f-4f77-4816-b3b0-bd90379b9548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.412000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613995f-4f77-4816-b3b0-bd90379b9548 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.413335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5e4dd-5a26-4c53-b836-5765304b75f3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.418507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b6db3-d2a7-4a90-b233-0c059eca7780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.419518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b6db3-d2a7-4a90-b233-0c059eca7780 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.419808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b6db3-d2a7-4a90-b233-0c059eca7780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.420264 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b6db3-d2a7-4a90-b233-0c059eca7780 admin admin] PATH_INFO: `/v3/projects/2ddfd22e3dde4cc89c073bd2d88d648b/users/ff944e4cf29b4297acc63b591d90daaf/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.424275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 800/1601] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:15 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:15.424748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613995f-4f77-4816-b3b0-bd90379b9548 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.437934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613995f-4f77-4816-b3b0-bd90379b9548 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.440179 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b6db3-d2a7-4a90-b233-0c059eca7780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ddfd22e3dde4cc89c073bd2d88d648b, user_id=ff944e4cf29b4297acc63b591d90daaf, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.443561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b6db3-d2a7-4a90-b233-0c059eca7780 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.447701 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9c93c-d413-453b-aec2-e90be8c2d198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.448620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c93c-d413-453b-aec2-e90be8c2d198 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.448961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c93c-d413-453b-aec2-e90be8c2d198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.449248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9c93c-d413-453b-aec2-e90be8c2d198 admin admin] PATH_INFO: `/v3/projects/f5b224b67d8641a184a061a1d2373a6c/users/a6bfdd7350d244edb8f5b09034c8f4a5/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.464538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 802/1602] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:15 2022] PUT /identity/v3/projects/2ddfd22e3dde4cc89c073bd2d88d648b/users/ff944e4cf29b4297acc63b591d90daaf/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:15.464538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b6db3-d2a7-4a90-b233-0c059eca7780 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.467414 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9c93c-d413-453b-aec2-e90be8c2d198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5b224b67d8641a184a061a1d2373a6c, user_id=a6bfdd7350d244edb8f5b09034c8f4a5, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.475060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9c93c-d413-453b-aec2-e90be8c2d198 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.497952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 801/1603] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:15 2022] PUT /identity/v3/projects/f5b224b67d8641a184a061a1d2373a6c/users/a6bfdd7350d244edb8f5b09034c8f4a5/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:15.508012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9c93c-d413-453b-aec2-e90be8c2d198 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.543422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b6db3-d2a7-4a90-b233-0c059eca7780 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.587544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9c93c-d413-453b-aec2-e90be8c2d198 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.593427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47af43-b17e-4406-b808-3e77904b57d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.594436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47af43-b17e-4406-b808-3e77904b57d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.594853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47af43-b17e-4406-b808-3e77904b57d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.595133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e47af43-b17e-4406-b808-3e77904b57d4 admin admin] PATH_INFO: `/v3/projects/92bf8a2fea6243c28aff53d39a57f92c/users/a7e574dc676841dea1f23a06c333ff14/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.612374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47af43-b17e-4406-b808-3e77904b57d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92bf8a2fea6243c28aff53d39a57f92c, user_id=a7e574dc676841dea1f23a06c333ff14, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.615338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e47af43-b17e-4406-b808-3e77904b57d4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.634004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 802/1604] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:15 2022] PUT /identity/v3/projects/92bf8a2fea6243c28aff53d39a57f92c/users/a7e574dc676841dea1f23a06c333ff14/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:15.635116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47af43-b17e-4406-b808-3e77904b57d4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.651300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818ce085-f2fd-44da-85f9-f7b80a7d8c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.652215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ce085-f2fd-44da-85f9-f7b80a7d8c38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.652507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ce085-f2fd-44da-85f9-f7b80a7d8c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.652761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818ce085-f2fd-44da-85f9-f7b80a7d8c38 admin admin] PATH_INFO: `/v3/projects/87aefc781c404b0cad8df35e450704b3/users/f483dacae8a94648aa2ee3d16e370b4e/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.679012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ce085-f2fd-44da-85f9-f7b80a7d8c38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87aefc781c404b0cad8df35e450704b3, user_id=f483dacae8a94648aa2ee3d16e370b4e, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.680795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:15.681997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818ce085-f2fd-44da-85f9-f7b80a7d8c38 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.706074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:15.713235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 803/1605] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:15 2022] PUT /identity/v3/projects/87aefc781c404b0cad8df35e450704b3/users/f483dacae8a94648aa2ee3d16e370b4e/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:15.716381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236189d-93e8-4975-a5fd-227e8758cf92 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.716648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236189d-93e8-4975-a5fd-227e8758cf92 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.716918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236189d-93e8-4975-a5fd-227e8758cf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.720739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e47af43-b17e-4406-b808-3e77904b57d4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.736180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a236189d-93e8-4975-a5fd-227e8758cf92 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:15.737413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a236189d-93e8-4975-a5fd-227e8758cf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:15.808017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a56dd0-c679-451c-8dd1-4b9f684330d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.811430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dd0-c679-451c-8dd1-4b9f684330d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.813190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dd0-c679-451c-8dd1-4b9f684330d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.813501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a56dd0-c679-451c-8dd1-4b9f684330d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.814232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a56dd0-c679-451c-8dd1-4b9f684330d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.816729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 804/1606] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:15.817468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a56dd0-c679-451c-8dd1-4b9f684330d9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.818049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236189d-93e8-4975-a5fd-227e8758cf92 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:15.829500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236189d-93e8-4975-a5fd-227e8758cf92 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:15.834469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8fb1c-fd0f-4238-b13d-453a23a9c9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:15.835824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fb1c-fd0f-4238-b13d-453a23a9c9f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.836282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fb1c-fd0f-4238-b13d-453a23a9c9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.836553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8fb1c-fd0f-4238-b13d-453a23a9c9f9 admin admin] PATH_INFO: `/v3/projects/e84fe8cfef0c4aa3a55dccc8f9aaddc9/users/90438171f42e4d74bd4e73bfb39ac91d/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.854306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8fb1c-fd0f-4238-b13d-453a23a9c9f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84fe8cfef0c4aa3a55dccc8f9aaddc9, user_id=90438171f42e4d74bd4e73bfb39ac91d, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:15.857622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8fb1c-fd0f-4238-b13d-453a23a9c9f9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:15.867331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 803/1607] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:15 2022] POST /identity/v3/projects => generated 379 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:15.869535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610950-e840-404e-bb5c-7be652d9dbfb None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.870267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610950-e840-404e-bb5c-7be652d9dbfb None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.870701 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19610950-e840-404e-bb5c-7be652d9dbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.877146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 805/1608] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:15 2022] PUT /identity/v3/projects/e84fe8cfef0c4aa3a55dccc8f9aaddc9/users/90438171f42e4d74bd4e73bfb39ac91d/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:15.878469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90178de8-f66d-4b25-ae98-675de93c967d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.878751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90178de8-f66d-4b25-ae98-675de93c967d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.879028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90178de8-f66d-4b25-ae98-675de93c967d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.897032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-19610950-e840-404e-bb5c-7be652d9dbfb None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:15.899947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-19610950-e840-404e-bb5c-7be652d9dbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:15.913146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-90178de8-f66d-4b25-ae98-675de93c967d None None] MFA Rules not processed for user `a6bfdd7350d244edb8f5b09034c8f4a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:15.914500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-90178de8-f66d-4b25-ae98-675de93c967d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:15.975479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 804/1609] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:15.977238 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead593-ca45-417a-aeb6-36d3fc79058d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.977744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead593-ca45-417a-aeb6-36d3fc79058d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.978465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead593-ca45-417a-aeb6-36d3fc79058d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:15.988424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 806/1610] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:15.990269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eecae-40d5-423c-bce5-c5bcdf7a79c6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:15.990555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eecae-40d5-423c-bce5-c5bcdf7a79c6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:15.990834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8eecae-40d5-423c-bce5-c5bcdf7a79c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.019221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ffead593-ca45-417a-aeb6-36d3fc79058d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:16.020803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ffead593-ca45-417a-aeb6-36d3fc79058d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:16.024331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2d8eecae-40d5-423c-bce5-c5bcdf7a79c6 None None] MFA Rules not processed for user `f483dacae8a94648aa2ee3d16e370b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:16.025862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2d8eecae-40d5-423c-bce5-c5bcdf7a79c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:16.042122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 805/1611] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:16.043389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffead593-ca45-417a-aeb6-36d3fc79058d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.091292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:16.102229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 807/1612] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:15 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:16.103597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8eecae-40d5-423c-bce5-c5bcdf7a79c6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.116710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8eecae-40d5-423c-bce5-c5bcdf7a79c6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.122578 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9044cd33-1f27-4245-98a9-380f5bab2534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.123909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044cd33-1f27-4245-98a9-380f5bab2534 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.124548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044cd33-1f27-4245-98a9-380f5bab2534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.125515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9044cd33-1f27-4245-98a9-380f5bab2534 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.126428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044cd33-1f27-4245-98a9-380f5bab2534 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.128871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9044cd33-1f27-4245-98a9-380f5bab2534 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.129282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffead593-ca45-417a-aeb6-36d3fc79058d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.133876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3825e-dd08-4c35-9b3b-46f31b944433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.134801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3825e-dd08-4c35-9b3b-46f31b944433 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.136222 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3825e-dd08-4c35-9b3b-46f31b944433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.136714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af3825e-dd08-4c35-9b3b-46f31b944433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.151797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3825e-dd08-4c35-9b3b-46f31b944433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.153381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af3825e-dd08-4c35-9b3b-46f31b944433 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.155561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 808/1613] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:16 2022] POST /identity/v3/users => generated 296 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:16.157294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67449f0-d800-4326-b4b0-62c332fabe09 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.157757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67449f0-d800-4326-b4b0-62c332fabe09 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.158338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67449f0-d800-4326-b4b0-62c332fabe09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.162859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:16.170793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 806/1614] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:16.171897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3825e-dd08-4c35-9b3b-46f31b944433 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.182306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af3825e-dd08-4c35-9b3b-46f31b944433 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.184331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b67449f0-d800-4326-b4b0-62c332fabe09 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:16.186187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b67449f0-d800-4326-b4b0-62c332fabe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:16.195968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 809/1615] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:16.198010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060006a9-56b1-41a5-acfb-0d01cca5d18e None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.198682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060006a9-56b1-41a5-acfb-0d01cca5d18e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.199158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060006a9-56b1-41a5-acfb-0d01cca5d18e None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.201788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 810/1616] 10.222.0.69 () {54 vars in 961 bytes} [Fri May 20 09:13:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:16.203146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060006a9-56b1-41a5-acfb-0d01cca5d18e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.206171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2793d-8dad-463e-b98c-bd50d4c452eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.207319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2793d-8dad-463e-b98c-bd50d4c452eb service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.207817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2793d-8dad-463e-b98c-bd50d4c452eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.208586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2793d-8dad-463e-b98c-bd50d4c452eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.218632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060006a9-56b1-41a5-acfb-0d01cca5d18e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.223896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccf82-5d0b-4185-ad8d-034265b11564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.225128 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccf82-5d0b-4185-ad8d-034265b11564 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.225705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccf82-5d0b-4185-ad8d-034265b11564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.226906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afccf82-5d0b-4185-ad8d-034265b11564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.241737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2793d-8dad-463e-b98c-bd50d4c452eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.249096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afccf82-5d0b-4185-ad8d-034265b11564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.250720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afccf82-5d0b-4185-ad8d-034265b11564 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.250862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2793d-8dad-463e-b98c-bd50d4c452eb service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.263417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 811/1617] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:16.264496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccf82-5d0b-4185-ad8d-034265b11564 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.276236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afccf82-5d0b-4185-ad8d-034265b11564 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.279018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 807/1618] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:16.280126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2793d-8dad-463e-b98c-bd50d4c452eb service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.282504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadfc2a-91d5-4404-8e94-0e41dc4d3bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.283951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadfc2a-91d5-4404-8e94-0e41dc4d3bc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.284098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadfc2a-91d5-4404-8e94-0e41dc4d3bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.284467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadfc2a-91d5-4404-8e94-0e41dc4d3bc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.285320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadfc2a-91d5-4404-8e94-0e41dc4d3bc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.288136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadfc2a-91d5-4404-8e94-0e41dc4d3bc9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.304372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2793d-8dad-463e-b98c-bd50d4c452eb service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.316355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5a82d-be9d-4b8f-b043-82799a045afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.317780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5a82d-be9d-4b8f-b043-82799a045afe admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.318176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5a82d-be9d-4b8f-b043-82799a045afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.318454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e5a82d-be9d-4b8f-b043-82799a045afe admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.322443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5a82d-be9d-4b8f-b043-82799a045afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.336396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e5a82d-be9d-4b8f-b043-82799a045afe admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.336584 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 812/1619] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:16 2022] POST /identity/v3/projects => generated 405 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:16.342359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717aeb9-a8e6-49eb-86f7-a628428ba0ae None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.342678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717aeb9-a8e6-49eb-86f7-a628428ba0ae None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.342926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5717aeb9-a8e6-49eb-86f7-a628428ba0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.362097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 808/1620] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:16.364453 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5a82d-be9d-4b8f-b043-82799a045afe admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.377783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5717aeb9-a8e6-49eb-86f7-a628428ba0ae None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:16.380158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5717aeb9-a8e6-49eb-86f7-a628428ba0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:16.386621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e5a82d-be9d-4b8f-b043-82799a045afe admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.487191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7cae8-5e77-454d-a22b-24d3c6cc3ef9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.487191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7cae8-5e77-454d-a22b-24d3c6cc3ef9 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.487191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7cae8-5e77-454d-a22b-24d3c6cc3ef9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.487191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d7cae8-5e77-454d-a22b-24d3c6cc3ef9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.500030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 813/1621] 10.222.0.69 () {58 vars in 1058 bytes} [Fri May 20 09:13:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:16.501169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717aeb9-a8e6-49eb-86f7-a628428ba0ae None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.600855 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7cae8-5e77-454d-a22b-24d3c6cc3ef9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.602346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d7cae8-5e77-454d-a22b-24d3c6cc3ef9 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.620857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5717aeb9-a8e6-49eb-86f7-a628428ba0ae None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.623686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 809/1622] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:16.624201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7cae8-5e77-454d-a22b-24d3c6cc3ef9 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.637975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d7cae8-5e77-454d-a22b-24d3c6cc3ef9 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.638974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9bc72-aa4f-4c98-aa4f-7dbfa25ed0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.639978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9bc72-aa4f-4c98-aa4f-7dbfa25ed0fc admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.640334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9bc72-aa4f-4c98-aa4f-7dbfa25ed0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.641107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9bc72-aa4f-4c98-aa4f-7dbfa25ed0fc admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.642138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9bc72-aa4f-4c98-aa4f-7dbfa25ed0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.643568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f8bd2-b0cb-432b-bc98-66b3f1639036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.644193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9bc72-aa4f-4c98-aa4f-7dbfa25ed0fc admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.644750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f8bd2-b0cb-432b-bc98-66b3f1639036 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.645121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f8bd2-b0cb-432b-bc98-66b3f1639036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.645626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f8bd2-b0cb-432b-bc98-66b3f1639036 admin admin] PATH_INFO: `/v3/projects/69dab0cec47e4e91bfc04ed72f43e714/users/5a3436394f144b40a8c22b47bac3f680/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.664493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f8bd2-b0cb-432b-bc98-66b3f1639036 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69dab0cec47e4e91bfc04ed72f43e714, user_id=5a3436394f144b40a8c22b47bac3f680, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.667778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3f8bd2-b0cb-432b-bc98-66b3f1639036 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.679159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 814/1623] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:16 2022] POST /identity/v3/users => generated 309 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:16.680075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9bc72-aa4f-4c98-aa4f-7dbfa25ed0fc admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.696442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 810/1624] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:16 2022] PUT /identity/v3/projects/69dab0cec47e4e91bfc04ed72f43e714/users/5a3436394f144b40a8c22b47bac3f680/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:16.698216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f8bd2-b0cb-432b-bc98-66b3f1639036 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.734552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9bc72-aa4f-4c98-aa4f-7dbfa25ed0fc admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.756682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:16.847766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683ccbb-1c0b-4c5e-8569-bb38857b222d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.848961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683ccbb-1c0b-4c5e-8569-bb38857b222d service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.849622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683ccbb-1c0b-4c5e-8569-bb38857b222d service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.849622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683ccbb-1c0b-4c5e-8569-bb38857b222d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.863303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3f8bd2-b0cb-432b-bc98-66b3f1639036 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.875474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5cd30-ba6c-46a7-b994-8896c3ad268d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.876259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5cd30-ba6c-46a7-b994-8896c3ad268d admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.876549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5cd30-ba6c-46a7-b994-8896c3ad268d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.876846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c5cd30-ba6c-46a7-b994-8896c3ad268d admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.877814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5cd30-ba6c-46a7-b994-8896c3ad268d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.882176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c5cd30-ba6c-46a7-b994-8896c3ad268d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.886027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:16.901739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 811/1625] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:16.903034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5cd30-ba6c-46a7-b994-8896c3ad268d admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.916914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c5cd30-ba6c-46a7-b994-8896c3ad268d admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.922451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fb69f-4d9d-4560-809e-da8e1b799112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.923587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fb69f-4d9d-4560-809e-da8e1b799112 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:16.923941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fb69f-4d9d-4560-809e-da8e1b799112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:16.924363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850fb69f-4d9d-4560-809e-da8e1b799112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:16.925346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fb69f-4d9d-4560-809e-da8e1b799112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.927368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850fb69f-4d9d-4560-809e-da8e1b799112 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.935852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683ccbb-1c0b-4c5e-8569-bb38857b222d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:16.937497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683ccbb-1c0b-4c5e-8569-bb38857b222d service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:16.944577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 812/1626] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:16.945736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fb69f-4d9d-4560-809e-da8e1b799112 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.951979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 815/1627] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:16.964604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683ccbb-1c0b-4c5e-8569-bb38857b222d service nova] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:16.966259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850fb69f-4d9d-4560-809e-da8e1b799112 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:16.982672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768be09a-88bc-4725-a8cf-f5acd5672bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:16.990156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683ccbb-1c0b-4c5e-8569-bb38857b222d service nova] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:17.000803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768be09a-88bc-4725-a8cf-f5acd5672bc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.001950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768be09a-88bc-4725-a8cf-f5acd5672bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.002277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768be09a-88bc-4725-a8cf-f5acd5672bc9 admin admin] PATH_INFO: `/v3/projects/bcef802f169c4f59a75de8b138fc915d/users/2db41b8ef7c04be9a0a296bafc0985f5/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.004582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfca54-57fe-4518-94fb-43421f9550b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:17.006680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfca54-57fe-4518-94fb-43421f9550b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.006953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfca54-57fe-4518-94fb-43421f9550b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.016800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfca54-57fe-4518-94fb-43421f9550b3 admin admin] PATH_INFO: `/v3/projects/69dab0cec47e4e91bfc04ed72f43e714/users/5a3436394f144b40a8c22b47bac3f680/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.024232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfca54-57fe-4518-94fb-43421f9550b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69dab0cec47e4e91bfc04ed72f43e714, user_id=5a3436394f144b40a8c22b47bac3f680, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:17.030379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfca54-57fe-4518-94fb-43421f9550b3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:17.068387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 816/1628] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:16 2022] PUT /identity/v3/projects/69dab0cec47e4e91bfc04ed72f43e714/users/5a3436394f144b40a8c22b47bac3f680/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:17.087661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c62d9b-21d8-4c2e-8a60-41880d6baaa5 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.087949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c62d9b-21d8-4c2e-8a60-41880d6baaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.088229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c62d9b-21d8-4c2e-8a60-41880d6baaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.095649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768be09a-88bc-4725-a8cf-f5acd5672bc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcef802f169c4f59a75de8b138fc915d, user_id=2db41b8ef7c04be9a0a296bafc0985f5, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:17.102731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768be09a-88bc-4725-a8cf-f5acd5672bc9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:17.136537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 813/1629] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:16 2022] PUT /identity/v3/projects/bcef802f169c4f59a75de8b138fc915d/users/2db41b8ef7c04be9a0a296bafc0985f5/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:17.141950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768be09a-88bc-4725-a8cf-f5acd5672bc9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:17.146765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-36c62d9b-21d8-4c2e-8a60-41880d6baaa5 None None] MFA Rules not processed for user `5a3436394f144b40a8c22b47bac3f680`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:17.148386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-36c62d9b-21d8-4c2e-8a60-41880d6baaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:17.284123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 817/1630] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:17 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:17.296947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c62d9b-21d8-4c2e-8a60-41880d6baaa5 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:17.345926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:17.387440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768be09a-88bc-4725-a8cf-f5acd5672bc9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:17.399686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c62d9b-21d8-4c2e-8a60-41880d6baaa5 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:17.401294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabbaab5-f711-4ddc-a19f-4e2a70ef0b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:17.402306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbaab5-f711-4ddc-a19f-4e2a70ef0b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.402595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbaab5-f711-4ddc-a19f-4e2a70ef0b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.402874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabbaab5-f711-4ddc-a19f-4e2a70ef0b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.404148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbaab5-f711-4ddc-a19f-4e2a70ef0b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:17.408034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabbaab5-f711-4ddc-a19f-4e2a70ef0b39 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:17.410455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4460fe-0285-4c00-a02b-ef834d790faa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:17.411784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4460fe-0285-4c00-a02b-ef834d790faa service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.413415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4460fe-0285-4c00-a02b-ef834d790faa service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.413714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4460fe-0285-4c00-a02b-ef834d790faa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.452348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 814/1631] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:17.453963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12347c-48ee-4e59-b631-32360255c00e None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.454262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12347c-48ee-4e59-b631-32360255c00e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.454546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12347c-48ee-4e59-b631-32360255c00e None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.455074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4460fe-0285-4c00-a02b-ef834d790faa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:17.456141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 815/1632] 10.222.0.69 () {54 vars in 912 bytes} [Fri May 20 09:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:17.456788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12347c-48ee-4e59-b631-32360255c00e None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:17.462749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4460fe-0285-4c00-a02b-ef834d790faa service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:17.470751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12347c-48ee-4e59-b631-32360255c00e None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:17.477174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1829d4-dd3a-4290-808c-95209cbd77aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:17.478370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1829d4-dd3a-4290-808c-95209cbd77aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.478656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1829d4-dd3a-4290-808c-95209cbd77aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.478938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1829d4-dd3a-4290-808c-95209cbd77aa admin admin] PATH_INFO: `/v3/projects/bcef802f169c4f59a75de8b138fc915d/users/2db41b8ef7c04be9a0a296bafc0985f5/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.481347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1829d4-dd3a-4290-808c-95209cbd77aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcef802f169c4f59a75de8b138fc915d, user_id=2db41b8ef7c04be9a0a296bafc0985f5, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:17.485313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1829d4-dd3a-4290-808c-95209cbd77aa admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:17.500122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 816/1633] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:17 2022] PUT /identity/v3/projects/bcef802f169c4f59a75de8b138fc915d/users/2db41b8ef7c04be9a0a296bafc0985f5/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:17.502908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 818/1634] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:17 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:17.513853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b3399-58ea-42ea-87c9-ffc206f33980 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.514151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b3399-58ea-42ea-87c9-ffc206f33980 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.514261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab763d-d781-4d53-a7aa-5f7dec5c1fd0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.514473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b3399-58ea-42ea-87c9-ffc206f33980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.514634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab763d-d781-4d53-a7aa-5f7dec5c1fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.514922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab763d-d781-4d53-a7aa-5f7dec5c1fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.538636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-985b3399-58ea-42ea-87c9-ffc206f33980 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:17.540169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-985b3399-58ea-42ea-87c9-ffc206f33980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:17.578137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3eab763d-d781-4d53-a7aa-5f7dec5c1fd0 None None] MFA Rules not processed for user `2db41b8ef7c04be9a0a296bafc0985f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:17.580631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3eab763d-d781-4d53-a7aa-5f7dec5c1fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:17.672531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 817/1635] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:13:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:17.693080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b3399-58ea-42ea-87c9-ffc206f33980 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:17.813526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 819/1636] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:17 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:17.814603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab763d-d781-4d53-a7aa-5f7dec5c1fd0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:17.843823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:17.888998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b3399-58ea-42ea-87c9-ffc206f33980 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:17.903729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ac38f-8b17-47bb-ac3c-ae0054ee5d51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:17.904428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac38f-8b17-47bb-ac3c-ae0054ee5d51 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:17.904712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac38f-8b17-47bb-ac3c-ae0054ee5d51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:17.904978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310ac38f-8b17-47bb-ac3c-ae0054ee5d51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:17.927286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ac38f-8b17-47bb-ac3c-ae0054ee5d51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:17.933828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310ac38f-8b17-47bb-ac3c-ae0054ee5d51 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:17.967394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 818/1637] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:13:17.988252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab763d-d781-4d53-a7aa-5f7dec5c1fd0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:17.999499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fadb2-ef4a-40eb-a206-384592099b6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:18.000286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fadb2-ef4a-40eb-a206-384592099b6a service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:18.002155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fadb2-ef4a-40eb-a206-384592099b6a service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:18.002480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1fadb2-ef4a-40eb-a206-384592099b6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:18.145566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fadb2-ef4a-40eb-a206-384592099b6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:18.147103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1fadb2-ef4a-40eb-a206-384592099b6a service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:18.172487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 820/1638] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:17 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:18.640705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ac38f-8b17-47bb-ac3c-ae0054ee5d51 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:18.653806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310ac38f-8b17-47bb-ac3c-ae0054ee5d51 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:18.662846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc5fe1-9cb4-4568-a0c0-e3f2fceda449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:18.663662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc5fe1-9cb4-4568-a0c0-e3f2fceda449 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:18.663954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc5fe1-9cb4-4568-a0c0-e3f2fceda449 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:18.666059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc5fe1-9cb4-4568-a0c0-e3f2fceda449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:18.690681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc5fe1-9cb4-4568-a0c0-e3f2fceda449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:18.694839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc5fe1-9cb4-4568-a0c0-e3f2fceda449 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:18.733190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 819/1639] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:18 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:19.049005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fadb2-ef4a-40eb-a206-384592099b6a service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:19.062379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:19.176159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1fadb2-ef4a-40eb-a206-384592099b6a service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:19.184729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:19.186121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:19.186616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:19.187611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] PATH_INFO: `/v3/users/f483dacae8a94648aa2ee3d16e370b4e` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:19.189587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f483dacae8a94648aa2ee3d16e370b4e)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:19.191922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: WARNING py.warnings [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:13:19.191922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: warnings.warn(msg) May 20 09:13:19.191922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: May 20 09:13:19.213492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:19.404357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f483dacae8a94648aa2ee3d16e370b4e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:19.447464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f483dacae8a94648aa2ee3d16e370b4e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:19.449175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f483dacae8a94648aa2ee3d16e370b4e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:19.473451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f483dacae8a94648aa2ee3d16e370b4e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:19.474749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f483dacae8a94648aa2ee3d16e370b4e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:19.536177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 821/1640] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:19 2022] DELETE /identity/v3/users/f483dacae8a94648aa2ee3d16e370b4e => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:19.540596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a687-2d73-4a59-930d-b69690d02f6f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:19.540885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a687-2d73-4a59-930d-b69690d02f6f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:19.541162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058a687-2d73-4a59-930d-b69690d02f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:19.576793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0058a687-2d73-4a59-930d-b69690d02f6f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:19.578329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0058a687-2d73-4a59-930d-b69690d02f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:19.652093 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 820/1641] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:19.726161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:19.728277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:19.845092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf36f9ce-2327-44d2-8122-02c5af708b7f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:19.851152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34da986-dcd5-4a20-a22d-f72442db8673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:19.852531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34da986-dcd5-4a20-a22d-f72442db8673 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:19.852940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34da986-dcd5-4a20-a22d-f72442db8673 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:19.853420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34da986-dcd5-4a20-a22d-f72442db8673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:19.873078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34da986-dcd5-4a20-a22d-f72442db8673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:19.874935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34da986-dcd5-4a20-a22d-f72442db8673 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:19.889691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 822/1642] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:20.288976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3530d-148b-466d-b09e-894fb6df43e1 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:20.289468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3530d-148b-466d-b09e-894fb6df43e1 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:20.289740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3530d-148b-466d-b09e-894fb6df43e1 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:20.292363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 821/1643] 10.222.0.69 () {54 vars in 961 bytes} [Fri May 20 09:13:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:20.303700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c07ad-75e0-4ee8-95db-182ae7a4d714 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:20.304276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c07ad-75e0-4ee8-95db-182ae7a4d714 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:20.304276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c07ad-75e0-4ee8-95db-182ae7a4d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:20.346125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-964c07ad-75e0-4ee8-95db-182ae7a4d714 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:20.350601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-964c07ad-75e0-4ee8-95db-182ae7a4d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:20.426355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:20.534309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 823/1644] 10.222.0.69 () {58 vars in 1058 bytes} [Fri May 20 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:20.539091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ed0f-b38f-46d4-9a50-0da870e5006d None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:20.539420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ed0f-b38f-46d4-9a50-0da870e5006d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:20.540022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6ed0f-b38f-46d4-9a50-0da870e5006d None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:20.542353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 822/1645] 10.222.0.69 () {54 vars in 912 bytes} [Fri May 20 09:13:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:20.548870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f0ece-2ef3-43dd-a3a1-2622cf104a20 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:20.549191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f0ece-2ef3-43dd-a3a1-2622cf104a20 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:20.549428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f0ece-2ef3-43dd-a3a1-2622cf104a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:20.588356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2a21-0b4b-4cad-8c56-e09a0bbcfa3f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:20.588643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2a21-0b4b-4cad-8c56-e09a0bbcfa3f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:20.588924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fd2a21-0b4b-4cad-8c56-e09a0bbcfa3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:20.596048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-df9f0ece-2ef3-43dd-a3a1-2622cf104a20 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:20.598472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-df9f0ece-2ef3-43dd-a3a1-2622cf104a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:20.615675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 824/1646] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:20.630277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f0ece-2ef3-43dd-a3a1-2622cf104a20 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:20.632395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-df9f0ece-2ef3-43dd-a3a1-2622cf104a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:20.684746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-53fd2a21-0b4b-4cad-8c56-e09a0bbcfa3f None None] MFA Rules not processed for user `90438171f42e4d74bd4e73bfb39ac91d`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:20.686090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-53fd2a21-0b4b-4cad-8c56-e09a0bbcfa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:20.773261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f0ece-2ef3-43dd-a3a1-2622cf104a20 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:20.788395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0a186-2e57-4e13-836a-add79ca584ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:20.790372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0a186-2e57-4e13-836a-add79ca584ef service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:20.795127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0a186-2e57-4e13-836a-add79ca584ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:20.795581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0a186-2e57-4e13-836a-add79ca584ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:20.828211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0a186-2e57-4e13-836a-add79ca584ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:20.829806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a0a186-2e57-4e13-836a-add79ca584ef service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:20.862096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 825/1647] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:13:20.894992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 823/1648] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:20.921356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d78d-e096-4662-abd3-3da653edccd8 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:20.921988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d78d-e096-4662-abd3-3da653edccd8 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:20.922363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f40d78d-e096-4662-abd3-3da653edccd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:20.945808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1f40d78d-e096-4662-abd3-3da653edccd8 None None] MFA Rules not processed for user `90438171f42e4d74bd4e73bfb39ac91d`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:20.947333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1f40d78d-e096-4662-abd3-3da653edccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:20.960760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 826/1649] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:20.974035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd2a21-0b4b-4cad-8c56-e09a0bbcfa3f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:20.976335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-53fd2a21-0b4b-4cad-8c56-e09a0bbcfa3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:21.059441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fd2a21-0b4b-4cad-8c56-e09a0bbcfa3f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:21.137266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a640e2a-aa8a-45db-97a7-08a1b43d0383 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:21.137266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a640e2a-aa8a-45db-97a7-08a1b43d0383 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:21.137266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a640e2a-aa8a-45db-97a7-08a1b43d0383 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:21.137266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a640e2a-aa8a-45db-97a7-08a1b43d0383 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:21.190496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a640e2a-aa8a-45db-97a7-08a1b43d0383 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:21.193184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a640e2a-aa8a-45db-97a7-08a1b43d0383 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:21.238352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 824/1650] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:21.796330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40d78d-e096-4662-abd3-3da653edccd8 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:21.797912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1f40d78d-e096-4662-abd3-3da653edccd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:21.821409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f40d78d-e096-4662-abd3-3da653edccd8 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:21.831984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:21.833418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:21.833908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:21.834370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] PATH_INFO: `/v3/projects/87aefc781c404b0cad8df35e450704b3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:21.835860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87aefc781c404b0cad8df35e450704b3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:21.838846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:21.943914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87aefc781c404b0cad8df35e450704b3', 'request_id': 'req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:21.951934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87aefc781c404b0cad8df35e450704b3', 'request_id': 'req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:21.960267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87aefc781c404b0cad8df35e450704b3', 'request_id': 'req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:21.981555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] The token cache is being invalidate because project 87aefc781c404b0cad8df35e450704b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:21.982356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:21.989459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0ebb0d1-55b7-430d-8347-2a6a75d0df02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:21.996364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a640e2a-aa8a-45db-97a7-08a1b43d0383 service nova] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:21.996809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 827/1651] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:21 2022] DELETE /identity/v3/projects/87aefc781c404b0cad8df35e450704b3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:22.017678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a640e2a-aa8a-45db-97a7-08a1b43d0383 service nova] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:22.090016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:22.112391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:22.113645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:22.114061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:22.114477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] PATH_INFO: `/v3/users/2db41b8ef7c04be9a0a296bafc0985f5` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:22.116097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2db41b8ef7c04be9a0a296bafc0985f5)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:22.118507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:22.230659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2db41b8ef7c04be9a0a296bafc0985f5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:22.231623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2db41b8ef7c04be9a0a296bafc0985f5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:22.232283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2db41b8ef7c04be9a0a296bafc0985f5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:22.249756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2db41b8ef7c04be9a0a296bafc0985f5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:22.259654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-70692887-4145-4664-81b2-1531293d21f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2db41b8ef7c04be9a0a296bafc0985f5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:22.286807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 825/1652] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:21 2022] DELETE /identity/v3/users/2db41b8ef7c04be9a0a296bafc0985f5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:22.558363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22889-6e86-423b-b402-06722a46b98c None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:22.558685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22889-6e86-423b-b402-06722a46b98c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:22.559027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f22889-6e86-423b-b402-06722a46b98c None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:22.561167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 828/1653] 10.222.0.69 () {54 vars in 912 bytes} [Fri May 20 09:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:22.567536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdf532-d117-4791-be26-c7a44a208f9a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:22.567957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdf532-d117-4791-be26-c7a44a208f9a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:22.568255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befdf532-d117-4791-be26-c7a44a208f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:22.621001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-befdf532-d117-4791-be26-c7a44a208f9a None None] MFA Rules not processed for user `349e99511c7c4307b27dc90b01f2dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:22.622488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-befdf532-d117-4791-be26-c7a44a208f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:22.731549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 826/1654] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:13:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:22.794281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22889-6e86-423b-b402-06722a46b98c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:22.796360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-27f22889-6e86-423b-b402-06722a46b98c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:22.814291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f22889-6e86-423b-b402-06722a46b98c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:22.819885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa919037-166b-4fa2-8921-d2ba7aaed9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:22.820710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919037-166b-4fa2-8921-d2ba7aaed9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:22.821125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919037-166b-4fa2-8921-d2ba7aaed9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:22.821541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa919037-166b-4fa2-8921-d2ba7aaed9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:22.834526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa919037-166b-4fa2-8921-d2ba7aaed9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:22.836757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa919037-166b-4fa2-8921-d2ba7aaed9f1 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:22.848742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 829/1655] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:24.102197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eeb99-8a98-40ce-80ac-1eb15ac73038 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.102788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eeb99-8a98-40ce-80ac-1eb15ac73038 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.102788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57eeb99-8a98-40ce-80ac-1eb15ac73038 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.104212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 827/1656] 10.222.0.69 () {54 vars in 960 bytes} [Fri May 20 09:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:13:24.112468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34f056-61af-4840-b377-db47b1b6ac38 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.112997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34f056-61af-4840-b377-db47b1b6ac38 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.113067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b34f056-61af-4840-b377-db47b1b6ac38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.145778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-9b34f056-61af-4840-b377-db47b1b6ac38 None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:24.147222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9b34f056-61af-4840-b377-db47b1b6ac38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:24.171511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:24.174253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eeb99-8a98-40ce-80ac-1eb15ac73038 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:24.176870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e57eeb99-8a98-40ce-80ac-1eb15ac73038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:24.279847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 830/1657] 10.222.0.69 () {58 vars in 1057 bytes} [Fri May 20 09:13:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:24.280958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34f056-61af-4840-b377-db47b1b6ac38 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:24.295955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b34f056-61af-4840-b377-db47b1b6ac38 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:24.312361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db483d41-5003-429b-b03b-6ec65189beb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:24.313138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db483d41-5003-429b-b03b-6ec65189beb4 service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.313620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db483d41-5003-429b-b03b-6ec65189beb4 service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.313961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db483d41-5003-429b-b03b-6ec65189beb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.315871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-db483d41-5003-429b-b03b-6ec65189beb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:24.335705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57eeb99-8a98-40ce-80ac-1eb15ac73038 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:24.340642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:24.341902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.342077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.342332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] PATH_INFO: `/v3/projects/bcef802f169c4f59a75de8b138fc915d` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.344496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcef802f169c4f59a75de8b138fc915d)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:24.344721 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db483d41-5003-429b-b03b-6ec65189beb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:24.345568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: WARNING py.warnings [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 20 09:13:24.345568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: warnings.warn(msg) May 20 09:13:24.345568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: May 20 09:13:24.346356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db483d41-5003-429b-b03b-6ec65189beb4 service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:24.346764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:24.348562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:24.360284 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 831/1658] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:24.446875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcef802f169c4f59a75de8b138fc915d', 'request_id': 'req-94f2bc4b-1380-448d-9a62-a101a419e6d6'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:24.465847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcef802f169c4f59a75de8b138fc915d', 'request_id': 'req-94f2bc4b-1380-448d-9a62-a101a419e6d6'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:24.487880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcef802f169c4f59a75de8b138fc915d', 'request_id': 'req-94f2bc4b-1380-448d-9a62-a101a419e6d6'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:24.526232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] The token cache is being invalidate because project bcef802f169c4f59a75de8b138fc915d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:24.527550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:24.528505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94f2bc4b-1380-448d-9a62-a101a419e6d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:24.533912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 828/1659] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:24 2022] DELETE /identity/v3/projects/bcef802f169c4f59a75de8b138fc915d => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:13:24.535988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9751d8-8e6e-4243-89f1-0570da5e629d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.536287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9751d8-8e6e-4243-89f1-0570da5e629d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.536544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9751d8-8e6e-4243-89f1-0570da5e629d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.592770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5b9751d8-8e6e-4243-89f1-0570da5e629d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:24.595073 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5b9751d8-8e6e-4243-89f1-0570da5e629d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:24.596166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd747e5-1ea2-4962-8a4c-e807f5e243c4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.596483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd747e5-1ea2-4962-8a4c-e807f5e243c4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.596764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd747e5-1ea2-4962-8a4c-e807f5e243c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.657987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6dd747e5-1ea2-4962-8a4c-e807f5e243c4 None None] MFA Rules not processed for user `992571802b7d426f923e0f0bc41e28de`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:24.667724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6dd747e5-1ea2-4962-8a4c-e807f5e243c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:24.746540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 832/1660] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:24.754319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccd847-0f30-428b-9eeb-ba6991acd45e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.754885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccd847-0f30-428b-9eeb-ba6991acd45e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.754885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ccd847-0f30-428b-9eeb-ba6991acd45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.812677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-56ccd847-0f30-428b-9eeb-ba6991acd45e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:24.814188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-56ccd847-0f30-428b-9eeb-ba6991acd45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:24.834919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 833/1661] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:24.836903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccd847-0f30-428b-9eeb-ba6991acd45e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:24.869864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ccd847-0f30-428b-9eeb-ba6991acd45e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:24.879366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 829/1662] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:24.885139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37572-58c6-46cb-8627-3c71421b8c91 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.885449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37572-58c6-46cb-8627-3c71421b8c91 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.885802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac37572-58c6-46cb-8627-3c71421b8c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.899596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:24.901038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:24.901329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:24.903540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:24.907985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:24.908628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:24.912232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:24.912630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-7ac37572-58c6-46cb-8627-3c71421b8c91 None None] MFA Rules not processed for user `992571802b7d426f923e0f0bc41e28de`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:24.914246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7ac37572-58c6-46cb-8627-3c71421b8c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:24.930367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 830/1663] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:24.946090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac37572-58c6-46cb-8627-3c71421b8c91 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:24.948970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7ac37572-58c6-46cb-8627-3c71421b8c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:24.983836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 834/1664] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:24.984854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:25.046476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6f8b6-9465-4d1c-8269-6e4c39b8a892 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:25.067472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399095d2-28ee-440a-829e-12735c99e830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:25.080108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399095d2-28ee-440a-829e-12735c99e830 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:25.082102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399095d2-28ee-440a-829e-12735c99e830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:25.082337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399095d2-28ee-440a-829e-12735c99e830 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:25.084907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399095d2-28ee-440a-829e-12735c99e830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:25.089053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399095d2-28ee-440a-829e-12735c99e830 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:25.191891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 835/1665] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:24 2022] POST /identity/v3/projects => generated 403 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:25.195147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399095d2-28ee-440a-829e-12735c99e830 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:25.269996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac37572-58c6-46cb-8627-3c71421b8c91 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:25.319185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:25.498897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399095d2-28ee-440a-829e-12735c99e830 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:25.518404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68069520-63e0-4202-819b-069b07e12daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:25.519160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68069520-63e0-4202-819b-069b07e12daa admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:25.519475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68069520-63e0-4202-819b-069b07e12daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:25.519758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68069520-63e0-4202-819b-069b07e12daa admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:25.520473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68069520-63e0-4202-819b-069b07e12daa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:25.533165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68069520-63e0-4202-819b-069b07e12daa admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:25.553641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a51bb-3079-448d-a491-f12ec237919d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:25.554426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a51bb-3079-448d-a491-f12ec237919d service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:25.556838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a51bb-3079-448d-a491-f12ec237919d service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:25.557351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a51bb-3079-448d-a491-f12ec237919d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:25.570675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:25.623048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 836/1666] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:25 2022] POST /identity/v3/users => generated 302 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:25.625174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68069520-63e0-4202-819b-069b07e12daa admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:25.653300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68069520-63e0-4202-819b-069b07e12daa admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:25.670840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ddbcd-727e-4d0b-b973-c0a8eb0be460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:25.676240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ddbcd-727e-4d0b-b973-c0a8eb0be460 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:25.676240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ddbcd-727e-4d0b-b973-c0a8eb0be460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:25.679305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ddbcd-727e-4d0b-b973-c0a8eb0be460 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:25.680262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ddbcd-727e-4d0b-b973-c0a8eb0be460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:25.690831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ddbcd-727e-4d0b-b973-c0a8eb0be460 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:25.710786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 837/1667] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:25.716050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ddbcd-727e-4d0b-b973-c0a8eb0be460 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:25.737492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a51bb-3079-448d-a491-f12ec237919d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:25.743886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318a51bb-3079-448d-a491-f12ec237919d service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:25.744451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ddbcd-727e-4d0b-b973-c0a8eb0be460 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:25.758641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93ec5d8-1570-4425-9a25-ddc5d6ae30ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:25.759517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec5d8-1570-4425-9a25-ddc5d6ae30ef admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:25.760048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec5d8-1570-4425-9a25-ddc5d6ae30ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:25.760424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93ec5d8-1570-4425-9a25-ddc5d6ae30ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:25.762389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ec5d8-1570-4425-9a25-ddc5d6ae30ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:25.766765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93ec5d8-1570-4425-9a25-ddc5d6ae30ef admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:25.786573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 831/1668] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 842 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:25.816762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 838/1669] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:25.822128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a51bb-3079-448d-a491-f12ec237919d service nova] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:25.859619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a51bb-3079-448d-a491-f12ec237919d service nova] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:25.892013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:25.895772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:25.896169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:25.896572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] PATH_INFO: `/v3/projects/c951c8772bd641f9b9ffff5eec22d6a3/users/acdd9c8b45a34e61b4c1642966862a7b/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:25.943346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c951c8772bd641f9b9ffff5eec22d6a3, user_id=acdd9c8b45a34e61b4c1642966862a7b, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:25.946920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:25.977187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 832/1670] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:25 2022] PUT /identity/v3/projects/c951c8772bd641f9b9ffff5eec22d6a3/users/acdd9c8b45a34e61b4c1642966862a7b/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:25.982655 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaec3e-958e-4751-83b4-e1767d0db5e4 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:25.983096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaec3e-958e-4751-83b4-e1767d0db5e4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:25.983413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eaec3e-958e-4751-83b4-e1767d0db5e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:26.030937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-57eaec3e-958e-4751-83b4-e1767d0db5e4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:26.033365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-57eaec3e-958e-4751-83b4-e1767d0db5e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:26.124726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 839/1671] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:26.157944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:26.160180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:26.318955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f1ba45-e25a-4363-9055-1f85dcf891ef admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:26.331190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af90761-743f-4f1a-bacb-aca2a365bdb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:26.331973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90761-743f-4f1a-bacb-aca2a365bdb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:26.332341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90761-743f-4f1a-bacb-aca2a365bdb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:26.332653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af90761-743f-4f1a-bacb-aca2a365bdb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:26.369490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af90761-743f-4f1a-bacb-aca2a365bdb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:26.372444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af90761-743f-4f1a-bacb-aca2a365bdb0 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:26.403665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 833/1672] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:28.382036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed27e2-6c90-4b6a-b014-1de684b4b11d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:28.382545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed27e2-6c90-4b6a-b014-1de684b4b11d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:28.382617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed27e2-6c90-4b6a-b014-1de684b4b11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:28.418249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-07ed27e2-6c90-4b6a-b014-1de684b4b11d None None] MFA Rules not processed for user `992571802b7d426f923e0f0bc41e28de`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:28.420918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-07ed27e2-6c90-4b6a-b014-1de684b4b11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:28.620567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 840/1673] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:29.920011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8783a-1e64-482b-8a9e-3ee787ae9469 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:29.921149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8783a-1e64-482b-8a9e-3ee787ae9469 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:29.921661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac8783a-1e64-482b-8a9e-3ee787ae9469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:29.945759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a5cd-0417-4682-a55f-e89e9197c560 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:29.946586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a5cd-0417-4682-a55f-e89e9197c560 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:29.947018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60a5cd-0417-4682-a55f-e89e9197c560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:29.989405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2d60a5cd-0417-4682-a55f-e89e9197c560 None None] MFA Rules not processed for user `acdd9c8b45a34e61b4c1642966862a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:29.999090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2d60a5cd-0417-4682-a55f-e89e9197c560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:30.057255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-bac8783a-1e64-482b-8a9e-3ee787ae9469 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:30.067897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bac8783a-1e64-482b-8a9e-3ee787ae9469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:30.124388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 841/1674] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:30.130069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60a5cd-0417-4682-a55f-e89e9197c560 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:30.165231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60a5cd-0417-4682-a55f-e89e9197c560 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:30.173953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588546b-6dba-42c0-bc8d-033541c40a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:30.175041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588546b-6dba-42c0-bc8d-033541c40a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:30.175489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588546b-6dba-42c0-bc8d-033541c40a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:30.175987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588546b-6dba-42c0-bc8d-033541c40a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:30.177032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588546b-6dba-42c0-bc8d-033541c40a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:30.179385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588546b-6dba-42c0-bc8d-033541c40a31 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:30.186177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:30.192541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:30.260407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 842/1675] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:30 2022] POST /identity/v3/projects => generated 405 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:13:30.266727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588546b-6dba-42c0-bc8d-033541c40a31 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:30.310421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 834/1676] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:13:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:30.456664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588546b-6dba-42c0-bc8d-033541c40a31 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:30.480422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9be8b-c09d-47fe-9347-818e48a3d4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:30.481520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9be8b-c09d-47fe-9347-818e48a3d4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:30.482001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9be8b-c09d-47fe-9347-818e48a3d4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:30.482494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc9be8b-c09d-47fe-9347-818e48a3d4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:30.487291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9be8b-c09d-47fe-9347-818e48a3d4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:30.489614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc9be8b-c09d-47fe-9347-818e48a3d4f4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:30.553559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 843/1677] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:30 2022] POST /identity/v3/users => generated 309 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:30.555690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8783a-1e64-482b-8a9e-3ee787ae9469 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:30.578762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac8783a-1e64-482b-8a9e-3ee787ae9469 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:30.603149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282832b5-d962-420d-8abe-5296389bac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:30.603891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282832b5-d962-420d-8abe-5296389bac32 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:30.604175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282832b5-d962-420d-8abe-5296389bac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:30.604510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282832b5-d962-420d-8abe-5296389bac32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:30.605199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282832b5-d962-420d-8abe-5296389bac32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:30.606891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282832b5-d962-420d-8abe-5296389bac32 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:30.643932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 835/1678] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:30.648209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9be8b-c09d-47fe-9347-818e48a3d4f4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:30.668025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc9be8b-c09d-47fe-9347-818e48a3d4f4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:30.701391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:30.702485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:30.703335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:30.705425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] PATH_INFO: `/v3/projects/c72c4fd7f4894234b985a06c51bcf361/users/c091b532f31c40b990a8333d94e0ecb0/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:30.751434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72c4fd7f4894234b985a06c51bcf361, user_id=c091b532f31c40b990a8333d94e0ecb0, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:30.754512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:30.793567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 844/1679] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:30 2022] PUT /identity/v3/projects/c72c4fd7f4894234b985a06c51bcf361/users/c091b532f31c40b990a8333d94e0ecb0/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:30.797734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282832b5-d962-420d-8abe-5296389bac32 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:30.806284 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:30.811715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:30.839531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282832b5-d962-420d-8abe-5296389bac32 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:31.072122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:31.072829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:31.073089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:31.073365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:31.074126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:31.082338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:31.108866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7d965e-a8d9-45c9-a9fe-e841a0441277 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:31.122480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 836/1680] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:31.143935 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:31.146504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8073594-8bf3-4b24-bfd4-dcda3815acb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:31.147559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8073594-8bf3-4b24-bfd4-dcda3815acb5 service swift] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:31.147981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8073594-8bf3-4b24-bfd4-dcda3815acb5 service swift] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:31.148627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8073594-8bf3-4b24-bfd4-dcda3815acb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:31.149534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:31.202329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72fd1fd-100a-4fee-887e-6975c3e65a0a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:31.228324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:31.229518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:31.233269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:31.233652 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] PATH_INFO: `/v3/users/a6bfdd7350d244edb8f5b09034c8f4a5` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:31.247429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6bfdd7350d244edb8f5b09034c8f4a5)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:31.254588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:31.373400 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:31.409100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8073594-8bf3-4b24-bfd4-dcda3815acb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:31.412791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8073594-8bf3-4b24-bfd4-dcda3815acb5 service swift] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:31.423448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6bfdd7350d244edb8f5b09034c8f4a5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:31.427728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6bfdd7350d244edb8f5b09034c8f4a5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:31.428442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6bfdd7350d244edb8f5b09034c8f4a5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:31.437540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 845/1681] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:13:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 637 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:13:31.439317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8073594-8bf3-4b24-bfd4-dcda3815acb5 service swift] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:31.442165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6bfdd7350d244edb8f5b09034c8f4a5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:31.450591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6bfdd7350d244edb8f5b09034c8f4a5'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:31.456771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8073594-8bf3-4b24-bfd4-dcda3815acb5 service swift] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:31.479252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4784b66-98c6-4b32-995e-8dc6eae4032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:31.480313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784b66-98c6-4b32-995e-8dc6eae4032a admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:31.480697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784b66-98c6-4b32-995e-8dc6eae4032a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:31.481084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4784b66-98c6-4b32-995e-8dc6eae4032a admin admin] PATH_INFO: `/v3/projects/c72c4fd7f4894234b985a06c51bcf361/users/c091b532f31c40b990a8333d94e0ecb0/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:31.485631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4784b66-98c6-4b32-995e-8dc6eae4032a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72c4fd7f4894234b985a06c51bcf361, user_id=c091b532f31c40b990a8333d94e0ecb0, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:31.492101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4784b66-98c6-4b32-995e-8dc6eae4032a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:31.504713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 837/1682] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:31 2022] DELETE /identity/v3/users/a6bfdd7350d244edb8f5b09034c8f4a5 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:31.505865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:31.510746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:31.518692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 846/1683] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:31 2022] PUT /identity/v3/projects/c72c4fd7f4894234b985a06c51bcf361/users/c091b532f31c40b990a8333d94e0ecb0/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:31.521161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123f60-48df-454e-a5a1-04a8831d6934 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:31.521830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123f60-48df-454e-a5a1-04a8831d6934 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:31.522519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2123f60-48df-454e-a5a1-04a8831d6934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:31.564476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f2123f60-48df-454e-a5a1-04a8831d6934 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:31.565945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f2123f60-48df-454e-a5a1-04a8831d6934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:31.590439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:31.688138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 847/1684] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:31.700686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6c2e6-a8fa-41cb-8f2c-d4c05130bbd7 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:31.730862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:31.732355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:31.732355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:31.732523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] PATH_INFO: `/v3/users/de4e89c56a0542a9bb92e7da79a5ead4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:31.737285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=de4e89c56a0542a9bb92e7da79a5ead4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:31.739730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:31.792984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2123f60-48df-454e-a5a1-04a8831d6934 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:31.795065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f2123f60-48df-454e-a5a1-04a8831d6934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:31.967104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de4e89c56a0542a9bb92e7da79a5ead4'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:31.968108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de4e89c56a0542a9bb92e7da79a5ead4'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:31.968864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de4e89c56a0542a9bb92e7da79a5ead4'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:31.983401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de4e89c56a0542a9bb92e7da79a5ead4'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:32.008731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2123f60-48df-454e-a5a1-04a8831d6934 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:32.025573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ddfe72ac-2e5a-4904-a992-1fba29b1ed6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de4e89c56a0542a9bb92e7da79a5ead4'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:32.040074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:32.041228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:32.041718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:32.042299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:32.057179 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:32.096107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:32.098037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:32.123053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 838/1685] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:31 2022] DELETE /identity/v3/users/de4e89c56a0542a9bb92e7da79a5ead4 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:32.131499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb93383-e480-48f6-a19b-9ea05e6b94c4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:32.132055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb93383-e480-48f6-a19b-9ea05e6b94c4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:32.133521 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 848/1686] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:32.134464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb93383-e480-48f6-a19b-9ea05e6b94c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:32.197685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-beb93383-e480-48f6-a19b-9ea05e6b94c4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:32.201806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-beb93383-e480-48f6-a19b-9ea05e6b94c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:32.405678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 839/1687] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:32.774628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:32.777065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:32.802294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc32b0d-c93c-4928-be2b-9eee5e708a9b service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:32.988503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e16541f-bb5a-461c-9388-09ca0a8d2632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:32.989674 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16541f-bb5a-461c-9388-09ca0a8d2632 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:32.990109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16541f-bb5a-461c-9388-09ca0a8d2632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:32.990531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e16541f-bb5a-461c-9388-09ca0a8d2632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:33.020348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16541f-bb5a-461c-9388-09ca0a8d2632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:33.022125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e16541f-bb5a-461c-9388-09ca0a8d2632 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:33.059605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 849/1688] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:35.530960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb93383-e480-48f6-a19b-9ea05e6b94c4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:35.532694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-beb93383-e480-48f6-a19b-9ea05e6b94c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:35.586540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb93383-e480-48f6-a19b-9ea05e6b94c4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:35.613132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:35.620749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:35.621445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:35.621804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] PATH_INFO: `/v3/projects/f5b224b67d8641a184a061a1d2373a6c` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:35.623538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5b224b67d8641a184a061a1d2373a6c)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:35.631586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:35.807662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:35.815688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5b224b67d8641a184a061a1d2373a6c', 'request_id': 'req-69fff5ff-303c-41db-b9ba-06f5b4cbb516'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:35.830734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5b224b67d8641a184a061a1d2373a6c', 'request_id': 'req-69fff5ff-303c-41db-b9ba-06f5b4cbb516'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:35.848710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5b224b67d8641a184a061a1d2373a6c', 'request_id': 'req-69fff5ff-303c-41db-b9ba-06f5b4cbb516'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:35.868953 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] The token cache is being invalidate because project f5b224b67d8641a184a061a1d2373a6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:35.869486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:35.870416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69fff5ff-303c-41db-b9ba-06f5b4cbb516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:35.875145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 840/1689] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:35 2022] DELETE /identity/v3/projects/f5b224b67d8641a184a061a1d2373a6c => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:35.880431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b3106-7482-4ada-bd3d-758ba77d774c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:35.882213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b3106-7482-4ada-bd3d-758ba77d774c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:35.882710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136b3106-7482-4ada-bd3d-758ba77d774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:35.909975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-136b3106-7482-4ada-bd3d-758ba77d774c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:35.912240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-136b3106-7482-4ada-bd3d-758ba77d774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:35.978576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:36.134571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 850/1690] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:36.143328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59558405-123c-4a47-8988-cad4a2282499 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:36.144788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59558405-123c-4a47-8988-cad4a2282499 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:36.145739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59558405-123c-4a47-8988-cad4a2282499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:36.217523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-59558405-123c-4a47-8988-cad4a2282499 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:36.218824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-59558405-123c-4a47-8988-cad4a2282499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:36.233793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 841/1691] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:36.253314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b3106-7482-4ada-bd3d-758ba77d774c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:36.296053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136b3106-7482-4ada-bd3d-758ba77d774c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:36.308753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c3281f-a2e4-4b31-b7ad-1eb6da21f456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:36.310501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3281f-a2e4-4b31-b7ad-1eb6da21f456 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:36.311661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3281f-a2e4-4b31-b7ad-1eb6da21f456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:36.314334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3281f-a2e4-4b31-b7ad-1eb6da21f456 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:36.315380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3281f-a2e4-4b31-b7ad-1eb6da21f456 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:36.315896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3281f-a2e4-4b31-b7ad-1eb6da21f456 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:36.325892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c3281f-a2e4-4b31-b7ad-1eb6da21f456 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:36.365226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 851/1692] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:36.369807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59558405-123c-4a47-8988-cad4a2282499 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:36.386981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59558405-123c-4a47-8988-cad4a2282499 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:36.398288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a82d8-1582-48d9-9d20-505e3b9a6e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:36.399341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a82d8-1582-48d9-9d20-505e3b9a6e26 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:36.399659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a82d8-1582-48d9-9d20-505e3b9a6e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:36.400497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a82d8-1582-48d9-9d20-505e3b9a6e26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:36.401507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a82d8-1582-48d9-9d20-505e3b9a6e26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:36.406585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a82d8-1582-48d9-9d20-505e3b9a6e26 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:36.470616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bb058-058f-4c64-879a-b81ef28cf8b6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:36.471249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bb058-058f-4c64-879a-b81ef28cf8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:36.471799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bb058-058f-4c64-879a-b81ef28cf8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:36.492542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 842/1693] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:36 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:36.497600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a82d8-1582-48d9-9d20-505e3b9a6e26 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:36.505865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-0b8bb058-058f-4c64-879a-b81ef28cf8b6 None None] MFA Rules not processed for user `349e99511c7c4307b27dc90b01f2dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:36.511203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0b8bb058-058f-4c64-879a-b81ef28cf8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:36.827354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 852/1694] 10.222.0.44 () {58 vars in 1015 bytes} [Fri May 20 09:13:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:36.829077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff57dd-d869-44e1-98b6-3cf62e5a8c3e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:36.829704 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff57dd-d869-44e1-98b6-3cf62e5a8c3e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:36.830292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff57dd-d869-44e1-98b6-3cf62e5a8c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:36.934461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a82d8-1582-48d9-9d20-505e3b9a6e26 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:36.946274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-edff57dd-d869-44e1-98b6-3cf62e5a8c3e None None] MFA Rules not processed for user `bfaaa367491745708334c5f35331bf76`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:36.947549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bf8de-ee3a-4b29-86bb-05fbc777a1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:36.947691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-edff57dd-d869-44e1-98b6-3cf62e5a8c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:36.949730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bf8de-ee3a-4b29-86bb-05fbc777a1c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:36.952452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bf8de-ee3a-4b29-86bb-05fbc777a1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:36.952928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bf8de-ee3a-4b29-86bb-05fbc777a1c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:36.953823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bf8de-ee3a-4b29-86bb-05fbc777a1c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:36.974012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bf8de-ee3a-4b29-86bb-05fbc777a1c8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:37.045275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 843/1695] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:36 2022] POST /identity/v3/users => generated 283 bytes in 549 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:37.066013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bf8de-ee3a-4b29-86bb-05fbc777a1c8 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:37.100005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bf8de-ee3a-4b29-86bb-05fbc777a1c8 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:37.135706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002dbdf-fa19-4cb6-8868-81ff89b0eb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:37.136407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002dbdf-fa19-4cb6-8868-81ff89b0eb95 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:37.136689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002dbdf-fa19-4cb6-8868-81ff89b0eb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:37.136960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1002dbdf-fa19-4cb6-8868-81ff89b0eb95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:37.137768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002dbdf-fa19-4cb6-8868-81ff89b0eb95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:37.139529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1002dbdf-fa19-4cb6-8868-81ff89b0eb95 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:37.191050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 844/1696] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:37.192069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002dbdf-fa19-4cb6-8868-81ff89b0eb95 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:37.224513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 853/1697] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 397 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:37.226141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8fa9-94ea-452a-ad49-24312d74c1a9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:37.227763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8fa9-94ea-452a-ad49-24312d74c1a9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:37.228215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6f8fa9-94ea-452a-ad49-24312d74c1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:37.245602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1002dbdf-fa19-4cb6-8868-81ff89b0eb95 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:37.267241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:37.267997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:37.268266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:37.268542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:37.269301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:37.270944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:37.297315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 845/1698] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:37.302890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:37.306416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:37.327902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-dd6f8fa9-94ea-452a-ad49-24312d74c1a9 None None] MFA Rules not processed for user `a7e574dc676841dea1f23a06c333ff14`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:37.331941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-dd6f8fa9-94ea-452a-ad49-24312d74c1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:37.334532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c090a1-6396-4a02-8e94-f3319618d2e8 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:37.349165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99033-0cd3-44c9-8ed2-f8f222aa2b1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:37.349982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99033-0cd3-44c9-8ed2-f8f222aa2b1f service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:37.350179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99033-0cd3-44c9-8ed2-f8f222aa2b1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:37.350461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c99033-0cd3-44c9-8ed2-f8f222aa2b1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:37.378023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c99033-0cd3-44c9-8ed2-f8f222aa2b1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:37.379672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c99033-0cd3-44c9-8ed2-f8f222aa2b1f service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:37.394126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 846/1699] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:37.395004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99033-0cd3-44c9-8ed2-f8f222aa2b1f service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:37.412841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c99033-0cd3-44c9-8ed2-f8f222aa2b1f service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:37.422997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:37.423731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:37.424210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:37.424726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:37.443644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:37.445527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:37.461629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 847/1700] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:37.462282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:37.471237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:37.476368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:37.520189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 854/1701] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:37.521057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f8fa9-94ea-452a-ad49-24312d74c1a9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:37.606861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6f8fa9-94ea-452a-ad49-24312d74c1a9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:37.633278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8e5bf-2084-42d2-95ab-b12a891aa389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:37.644405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8e5bf-2084-42d2-95ab-b12a891aa389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:37.644405 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8e5bf-2084-42d2-95ab-b12a891aa389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:37.644774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8e5bf-2084-42d2-95ab-b12a891aa389 admin admin] PATH_INFO: `/v3/projects/06b573cd178a4483bd978ab79f920fed/users/6f968dfa96844dc1aee984351f91d1da/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:37.742567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8e5bf-2084-42d2-95ab-b12a891aa389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06b573cd178a4483bd978ab79f920fed, user_id=6f968dfa96844dc1aee984351f91d1da, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:37.754539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8e5bf-2084-42d2-95ab-b12a891aa389 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:37.757459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:37.790579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 855/1702] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:37 2022] PUT /identity/v3/projects/06b573cd178a4483bd978ab79f920fed/users/6f968dfa96844dc1aee984351f91d1da/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:37.794204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29fad9-c828-4f0b-89d7-f6356a8d725f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:37.795377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29fad9-c828-4f0b-89d7-f6356a8d725f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:37.795740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29fad9-c828-4f0b-89d7-f6356a8d725f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:37.869704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23c6cfd-9570-4ea5-9e7e-1fa83bcd56bd service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:37.893992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-cd29fad9-c828-4f0b-89d7-f6356a8d725f None None] MFA Rules not processed for user `ff944e4cf29b4297acc63b591d90daaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:37.897308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cd29fad9-c828-4f0b-89d7-f6356a8d725f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:38.152574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:38.155916 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:38.156822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:38.158499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:38.213893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 856/1703] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 423 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:38.217955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29fad9-c828-4f0b-89d7-f6356a8d725f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:38.220963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cd29fad9-c828-4f0b-89d7-f6356a8d725f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:38.286129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:38.290505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:38.305213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 848/1704] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:37 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 843 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:38.306483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:38.309264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:38.329304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314af73-643a-4641-87bf-60c41c1ad99b service nova] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:38.338557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8136c1a0-ff97-4af4-94ad-1d563e568885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:38.339567 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c1a0-ff97-4af4-94ad-1d563e568885 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:38.340014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c1a0-ff97-4af4-94ad-1d563e568885 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:38.340615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8136c1a0-ff97-4af4-94ad-1d563e568885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:38.455201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136c1a0-ff97-4af4-94ad-1d563e568885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:38.461086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8136c1a0-ff97-4af4-94ad-1d563e568885 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:38.491219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29fad9-c828-4f0b-89d7-f6356a8d725f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:38.514506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 849/1705] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:38.514783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:38.515806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:38.516085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:38.516358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] PATH_INFO: `/v3/projects/69203fd149f44cc7830ca471e186ac78` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:38.518362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69203fd149f44cc7830ca471e186ac78)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:38.520365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:38.521135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38865eb0-f951-467d-9b30-267a8a625f50 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:38.523086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38865eb0-f951-467d-9b30-267a8a625f50 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:38.523396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38865eb0-f951-467d-9b30-267a8a625f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:38.676255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-38865eb0-f951-467d-9b30-267a8a625f50 None None] MFA Rules not processed for user `6f968dfa96844dc1aee984351f91d1da`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:38.677502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-38865eb0-f951-467d-9b30-267a8a625f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:38.726378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:38.739349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:38.744217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69203fd149f44cc7830ca471e186ac78', 'request_id': 'req-a90be6d1-9dd2-4abc-81ab-280623343458'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:38.798634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69203fd149f44cc7830ca471e186ac78', 'request_id': 'req-a90be6d1-9dd2-4abc-81ab-280623343458'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:38.834961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69203fd149f44cc7830ca471e186ac78', 'request_id': 'req-a90be6d1-9dd2-4abc-81ab-280623343458'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:38.876032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] The token cache is being invalidate because project 69203fd149f44cc7830ca471e186ac78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:38.876477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:38.886924 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a90be6d1-9dd2-4abc-81ab-280623343458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:38.893888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 857/1706] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:38 2022] DELETE /identity/v3/projects/69203fd149f44cc7830ca471e186ac78 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:38.893888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55143a17-458a-47ac-aba6-cdef06f543c5 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:38.893888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55143a17-458a-47ac-aba6-cdef06f543c5 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:38.893888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55143a17-458a-47ac-aba6-cdef06f543c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:38.967618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 850/1707] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:38.969573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38865eb0-f951-467d-9b30-267a8a625f50 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:38.973476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-38865eb0-f951-467d-9b30-267a8a625f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:38.995673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-55143a17-458a-47ac-aba6-cdef06f543c5 None None] MFA Rules not processed for user `c091b532f31c40b990a8333d94e0ecb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:38.997022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-55143a17-458a-47ac-aba6-cdef06f543c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:39.156953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 858/1708] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:38 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:39.158034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55143a17-458a-47ac-aba6-cdef06f543c5 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:39.173260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-55143a17-458a-47ac-aba6-cdef06f543c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:39.280037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55143a17-458a-47ac-aba6-cdef06f543c5 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:39.280768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38865eb0-f951-467d-9b30-267a8a625f50 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:39.287160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:39.288168 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:39.288483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:39.289006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:39.289386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:39.289670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:39.289920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:39.290024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] PATH_INFO: `/v3/users/5a3436394f144b40a8c22b47bac3f680` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:39.290942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:39.293437 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:39.296640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a3436394f144b40a8c22b47bac3f680)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:39.301146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:39.345409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 859/1709] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:39 2022] POST /identity/v3/projects => generated 365 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:39.347659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:39.350607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:39.459503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a3436394f144b40a8c22b47bac3f680'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:39.460572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a3436394f144b40a8c22b47bac3f680'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:39.461361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a3436394f144b40a8c22b47bac3f680'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:39.489073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a3436394f144b40a8c22b47bac3f680'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:39.505668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a3436394f144b40a8c22b47bac3f680'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:39.511809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:39.552047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 851/1710] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:38 2022] DELETE /identity/v3/users/5a3436394f144b40a8c22b47bac3f680 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:39.555470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:39.557666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:39.569657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c348295-de16-4411-aada-cec2799edf93 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:39.638838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:39.747133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfb9db-97b7-47b0-843f-db780636d57d admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:39.755036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb8107-317f-41e4-ae97-215431bafeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:39.755852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb8107-317f-41e4-ae97-215431bafeb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:39.756055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb8107-317f-41e4-ae97-215431bafeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:39.756375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb8107-317f-41e4-ae97-215431bafeb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:39.757435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb8107-317f-41e4-ae97-215431bafeb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:39.759513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb8107-317f-41e4-ae97-215431bafeb5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:39.766181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:39.768916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:39.769455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:39.769893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:39.781980 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:39.840795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 852/1711] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:39 2022] POST /identity/v3/users => generated 289 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:39.842060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb8107-317f-41e4-ae97-215431bafeb5 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:39.893305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb8107-317f-41e4-ae97-215431bafeb5 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:39.910037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056c2d5-33f7-400b-8654-c5e4b0fcfa8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:39.912955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056c2d5-33f7-400b-8654-c5e4b0fcfa8c admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:39.913249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056c2d5-33f7-400b-8654-c5e4b0fcfa8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:39.913552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056c2d5-33f7-400b-8654-c5e4b0fcfa8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:39.917450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056c2d5-33f7-400b-8654-c5e4b0fcfa8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:39.919989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056c2d5-33f7-400b-8654-c5e4b0fcfa8c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:39.952283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:39.955926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:39.968193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 853/1712] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:39.969995 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056c2d5-33f7-400b-8654-c5e4b0fcfa8c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:39.989951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 860/1713] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:39.994957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056c2d5-33f7-400b-8654-c5e4b0fcfa8c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:40.017969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff8e7e-3cbd-4bc8-8ca3-a7d6fbb79a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:40.026458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff8e7e-3cbd-4bc8-8ca3-a7d6fbb79a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:40.026954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff8e7e-3cbd-4bc8-8ca3-a7d6fbb79a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:40.027634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff8e7e-3cbd-4bc8-8ca3-a7d6fbb79a7b admin admin] PATH_INFO: `/v3/projects/3cdc9d7c04474a868c926cf065427f5e/users/624bb13c2d094dc7820b3304ff0cc470/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:40.076368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff8e7e-3cbd-4bc8-8ca3-a7d6fbb79a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cdc9d7c04474a868c926cf065427f5e, user_id=624bb13c2d094dc7820b3304ff0cc470, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:40.079476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff8e7e-3cbd-4bc8-8ca3-a7d6fbb79a7b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:40.122133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 854/1714] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:39 2022] PUT /identity/v3/projects/3cdc9d7c04474a868c926cf065427f5e/users/624bb13c2d094dc7820b3304ff0cc470/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:40.126290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:40.142194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267dd100-4612-448a-9787-1743f1a2a821 service nova] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:40.300048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f675ab5-89b1-4999-99df-4ab37b6a721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:40.300927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f675ab5-89b1-4999-99df-4ab37b6a721c admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:40.301219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f675ab5-89b1-4999-99df-4ab37b6a721c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:40.301606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f675ab5-89b1-4999-99df-4ab37b6a721c admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:40.302359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f675ab5-89b1-4999-99df-4ab37b6a721c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:40.306205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f675ab5-89b1-4999-99df-4ab37b6a721c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:40.335158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 861/1715] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:13:40.336988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff8e7e-3cbd-4bc8-8ca3-a7d6fbb79a7b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:40.357553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff8e7e-3cbd-4bc8-8ca3-a7d6fbb79a7b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:40.377618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:40.379088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:40.379088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:40.379297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] PATH_INFO: `/v3/projects/3cdc9d7c04474a868c926cf065427f5e/users/624bb13c2d094dc7820b3304ff0cc470/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:40.382630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cdc9d7c04474a868c926cf065427f5e, user_id=624bb13c2d094dc7820b3304ff0cc470, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:40.388450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:40.415203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 855/1716] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:40 2022] PUT /identity/v3/projects/3cdc9d7c04474a868c926cf065427f5e/users/624bb13c2d094dc7820b3304ff0cc470/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:13:40.426389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a3b06-c02e-4385-a91f-683fe180202b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:40.426389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a3b06-c02e-4385-a91f-683fe180202b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:40.426854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a3b06-c02e-4385-a91f-683fe180202b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:40.478596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4c2a3b06-c02e-4385-a91f-683fe180202b None None] MFA Rules not processed for user `624bb13c2d094dc7820b3304ff0cc470`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:40.481435 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4c2a3b06-c02e-4385-a91f-683fe180202b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:40.496106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:40.655817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 862/1717] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:40.670057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:40.672244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:40.829041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182cd5a-7013-498b-9f48-ab4fe4334a00 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:40.840376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:40.842469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:40.842756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:40.843030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:40.872505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:40.876846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:40.916329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 856/1718] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:41.149659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a3b06-c02e-4385-a91f-683fe180202b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:41.173509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a3b06-c02e-4385-a91f-683fe180202b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:41.177146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3741b6f-1003-48b2-9227-f7e936531360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:41.179071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3741b6f-1003-48b2-9227-f7e936531360 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:41.179753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3741b6f-1003-48b2-9227-f7e936531360 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:41.180050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3741b6f-1003-48b2-9227-f7e936531360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:41.182036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a3741b6f-1003-48b2-9227-f7e936531360 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:41.410095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3741b6f-1003-48b2-9227-f7e936531360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:41.425175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3741b6f-1003-48b2-9227-f7e936531360 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:41.470446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 863/1719] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:41 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:44.025754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:44.028800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:44.180698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392075d6-f58b-4ce2-afd1-bfb201f02a76 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:44.188265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:44.189448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:44.189961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:44.190329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] PATH_INFO: `/v3/projects/69dab0cec47e4e91bfc04ed72f43e714` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:44.192221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69dab0cec47e4e91bfc04ed72f43e714)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:44.195038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:44.212209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:44.384073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69dab0cec47e4e91bfc04ed72f43e714', 'request_id': 'req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:44.398399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69dab0cec47e4e91bfc04ed72f43e714', 'request_id': 'req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:44.407355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69dab0cec47e4e91bfc04ed72f43e714', 'request_id': 'req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:44.427807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] The token cache is being invalidate because project 69dab0cec47e4e91bfc04ed72f43e714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:44.428105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:44.429339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e60a3e1-63f9-48b8-8365-9cb4005a4c33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:44.437972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83963c1-19dd-49fc-b5cc-8c6ef6a36bd9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:44.438602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83963c1-19dd-49fc-b5cc-8c6ef6a36bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:44.438602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83963c1-19dd-49fc-b5cc-8c6ef6a36bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:44.464112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 857/1720] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:44 2022] DELETE /identity/v3/projects/69dab0cec47e4e91bfc04ed72f43e714 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:44.484206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d83963c1-19dd-49fc-b5cc-8c6ef6a36bd9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:44.486574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d83963c1-19dd-49fc-b5cc-8c6ef6a36bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:44.726092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 864/1721] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:44.737104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd438053-41a5-440d-ac5f-937e94b782bc None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:44.737655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd438053-41a5-440d-ac5f-937e94b782bc None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:44.738096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd438053-41a5-440d-ac5f-937e94b782bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:44.804322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-fd438053-41a5-440d-ac5f-937e94b782bc None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:44.805091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fd438053-41a5-440d-ac5f-937e94b782bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:44.838730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 858/1722] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:44.845195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83963c1-19dd-49fc-b5cc-8c6ef6a36bd9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:44.859157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83963c1-19dd-49fc-b5cc-8c6ef6a36bd9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:44.878491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:44.879116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:44.879432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:44.879711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:44.881285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:44.881285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:44.882422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:44.940616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 865/1723] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:44.945048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd438053-41a5-440d-ac5f-937e94b782bc None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:44.958031 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd438053-41a5-440d-ac5f-937e94b782bc None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:44.963213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76586557-acdb-42a7-87d0-b03b0284fd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:44.963953 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76586557-acdb-42a7-87d0-b03b0284fd7c admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:44.964233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76586557-acdb-42a7-87d0-b03b0284fd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:44.964512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76586557-acdb-42a7-87d0-b03b0284fd7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:44.965616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76586557-acdb-42a7-87d0-b03b0284fd7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:44.967611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76586557-acdb-42a7-87d0-b03b0284fd7c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:45.026902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 859/1724] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:44 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:45.032388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:45.074636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bf794a-a89b-4612-bc77-6eeb7a12d0df admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:45.152904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:45.226403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18913a-5164-48c1-b923-ef7233993195 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:45.227420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18913a-5164-48c1-b923-ef7233993195 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:45.227680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18913a-5164-48c1-b923-ef7233993195 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:45.234362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 860/1725] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:13:45 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:45.331387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db05fa-4d37-45a1-9ec9-12382b6ebbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:45.333069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db05fa-4d37-45a1-9ec9-12382b6ebbab admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:45.333368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db05fa-4d37-45a1-9ec9-12382b6ebbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:45.333648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db05fa-4d37-45a1-9ec9-12382b6ebbab admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:45.334355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db05fa-4d37-45a1-9ec9-12382b6ebbab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:45.365170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db05fa-4d37-45a1-9ec9-12382b6ebbab admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:45.399310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18913a-5164-48c1-b923-ef7233993195 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:45.404205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1c18913a-5164-48c1-b923-ef7233993195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:45.439982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 866/1726] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:45 2022] POST /identity/v3/users => generated 288 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:45.441320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db05fa-4d37-45a1-9ec9-12382b6ebbab admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:45.444139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18913a-5164-48c1-b923-ef7233993195 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:45.478417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:45.482883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:45.483294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:45.483588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] PATH_INFO: `/v3/users/6f968dfa96844dc1aee984351f91d1da` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:45.484269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db05fa-4d37-45a1-9ec9-12382b6ebbab admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:45.486007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f968dfa96844dc1aee984351f91d1da)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:45.492038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:45.496547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdad43-2ab2-4d02-afae-622530efd9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:45.497741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdad43-2ab2-4d02-afae-622530efd9cf admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:45.499453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdad43-2ab2-4d02-afae-622530efd9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:45.499736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdad43-2ab2-4d02-afae-622530efd9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:45.501423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdad43-2ab2-4d02-afae-622530efd9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:45.506920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acdad43-2ab2-4d02-afae-622530efd9cf admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:45.539948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 867/1727] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:45.541097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdad43-2ab2-4d02-afae-622530efd9cf admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:45.563972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdad43-2ab2-4d02-afae-622530efd9cf admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:45.576591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:45.577338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:45.577620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:45.578039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:45.578924 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:45.607636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:45.622968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f968dfa96844dc1aee984351f91d1da'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:45.623751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f968dfa96844dc1aee984351f91d1da'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:45.624443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f968dfa96844dc1aee984351f91d1da'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:45.640256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f968dfa96844dc1aee984351f91d1da'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:45.653431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 868/1728] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:45.654518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:45.662008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:45.680080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-dfcb144d-5f2b-4313-aca6-20b991ea8091 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f968dfa96844dc1aee984351f91d1da'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:45.687102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476445d-a014-4921-9de5-e4245fe2f591 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:45.702776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56c3e1-08ea-4cd7-a466-191ad8743c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:45.703730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56c3e1-08ea-4cd7-a466-191ad8743c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:45.704081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56c3e1-08ea-4cd7-a466-191ad8743c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:45.705625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56c3e1-08ea-4cd7-a466-191ad8743c55 admin admin] PATH_INFO: `/v3/projects/d0bcb0fa0fd644feaad5ffe766b47c04/users/3289ac584c134566b6378c1de33f669c/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:45.738831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 861/1729] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:45 2022] DELETE /identity/v3/users/6f968dfa96844dc1aee984351f91d1da => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:13:45.745642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269a78-b058-44d9-aaca-dc4de1837f01 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:45.745972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269a78-b058-44d9-aaca-dc4de1837f01 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:45.746209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64269a78-b058-44d9-aaca-dc4de1837f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:45.748919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56c3e1-08ea-4cd7-a466-191ad8743c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0bcb0fa0fd644feaad5ffe766b47c04, user_id=3289ac584c134566b6378c1de33f669c, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:45.752728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56c3e1-08ea-4cd7-a466-191ad8743c55 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:45.772738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 869/1730] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:45 2022] PUT /identity/v3/projects/d0bcb0fa0fd644feaad5ffe766b47c04/users/3289ac584c134566b6378c1de33f669c/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:45.786105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515284d6-f3b4-4a8c-89a0-e34daca8968e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:45.786890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515284d6-f3b4-4a8c-89a0-e34daca8968e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:45.787906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515284d6-f3b4-4a8c-89a0-e34daca8968e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:45.812561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-64269a78-b058-44d9-aaca-dc4de1837f01 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:45.813653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-64269a78-b058-44d9-aaca-dc4de1837f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:45.848991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-515284d6-f3b4-4a8c-89a0-e34daca8968e None None] MFA Rules not processed for user `3289ac584c134566b6378c1de33f669c`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:45.852203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-515284d6-f3b4-4a8c-89a0-e34daca8968e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:45.919002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:45.964234 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 862/1731] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:45.985596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 870/1732] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:45.990091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64269a78-b058-44d9-aaca-dc4de1837f01 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:46.006667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64269a78-b058-44d9-aaca-dc4de1837f01 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:46.018283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbc45a-5e18-4db1-a0e8-f8863f2c590d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:46.018812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbc45a-5e18-4db1-a0e8-f8863f2c590d admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.019095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbc45a-5e18-4db1-a0e8-f8863f2c590d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.020469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dbc45a-5e18-4db1-a0e8-f8863f2c590d admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.021727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbc45a-5e18-4db1-a0e8-f8863f2c590d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:46.030007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dbc45a-5e18-4db1-a0e8-f8863f2c590d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:46.035685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e223-ea68-4d6a-a382-9de084be50bd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.036634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e223-ea68-4d6a-a382-9de084be50bd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.037145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6e223-ea68-4d6a-a382-9de084be50bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.063272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f6f6e223-ea68-4d6a-a382-9de084be50bd None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:46.064654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f6f6e223-ea68-4d6a-a382-9de084be50bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:46.096149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 863/1733] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:45 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:46.100233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbc45a-5e18-4db1-a0e8-f8863f2c590d admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:46.210403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:46.258407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dbc45a-5e18-4db1-a0e8-f8863f2c590d admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:46.264684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774e69b-4970-4534-8e9e-02e0d20c88ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:46.265744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774e69b-4970-4534-8e9e-02e0d20c88ce admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.266091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774e69b-4970-4534-8e9e-02e0d20c88ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.266461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774e69b-4970-4534-8e9e-02e0d20c88ce admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.268353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774e69b-4970-4534-8e9e-02e0d20c88ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:46.270498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774e69b-4970-4534-8e9e-02e0d20c88ce admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:46.275238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 871/1734] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:46.276480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6e223-ea68-4d6a-a382-9de084be50bd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:46.280228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f6f6e223-ea68-4d6a-a382-9de084be50bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:46.309398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 864/1735] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:46 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:46.311149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774e69b-4970-4534-8e9e-02e0d20c88ce admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:46.336352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774e69b-4970-4534-8e9e-02e0d20c88ce admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:46.351515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4acb58-434d-4ed8-857b-274c6be6728c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:46.352464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4acb58-434d-4ed8-857b-274c6be6728c admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.352841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4acb58-434d-4ed8-857b-274c6be6728c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.353253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4acb58-434d-4ed8-857b-274c6be6728c admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.354118 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4acb58-434d-4ed8-857b-274c6be6728c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:46.357140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4acb58-434d-4ed8-857b-274c6be6728c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:46.385108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 865/1736] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:46.389820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4acb58-434d-4ed8-857b-274c6be6728c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:46.401840 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4acb58-434d-4ed8-857b-274c6be6728c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:46.407515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55046a90-07cf-40a7-9608-0b45df659539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:46.408519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55046a90-07cf-40a7-9608-0b45df659539 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.408867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55046a90-07cf-40a7-9608-0b45df659539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.409207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55046a90-07cf-40a7-9608-0b45df659539 admin admin] PATH_INFO: `/v3/projects/e9579225e79b44babbfdba246b229acb/users/258361576e804fb596e22af3d12e7856/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.441429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55046a90-07cf-40a7-9608-0b45df659539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9579225e79b44babbfdba246b229acb, user_id=258361576e804fb596e22af3d12e7856, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:46.449346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55046a90-07cf-40a7-9608-0b45df659539 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:46.464140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6e223-ea68-4d6a-a382-9de084be50bd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:46.496353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 866/1737] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:46 2022] PUT /identity/v3/projects/e9579225e79b44babbfdba246b229acb/users/258361576e804fb596e22af3d12e7856/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:46.499107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55046a90-07cf-40a7-9608-0b45df659539 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:46.629051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea125c1-b3ce-41cc-8096-d5afd68dfc92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:46.630880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea125c1-b3ce-41cc-8096-d5afd68dfc92 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.631232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea125c1-b3ce-41cc-8096-d5afd68dfc92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.631530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea125c1-b3ce-41cc-8096-d5afd68dfc92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.653661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55046a90-07cf-40a7-9608-0b45df659539 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:46.661554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5be6705-0272-4f4f-a5b3-538fc365f95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:46.662465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be6705-0272-4f4f-a5b3-538fc365f95b admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.662920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be6705-0272-4f4f-a5b3-538fc365f95b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.663402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5be6705-0272-4f4f-a5b3-538fc365f95b admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.664359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be6705-0272-4f4f-a5b3-538fc365f95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:46.666462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5be6705-0272-4f4f-a5b3-538fc365f95b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:46.676127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea125c1-b3ce-41cc-8096-d5afd68dfc92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:46.681347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea125c1-b3ce-41cc-8096-d5afd68dfc92 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:46.681911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 867/1738] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:46.683682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a53a6-bf60-4af3-a9e4-4278da43011a None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.684177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a53a6-bf60-4af3-a9e4-4278da43011a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.684673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166a53a6-bf60-4af3-a9e4-4278da43011a None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.686768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 868/1739] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:46.687758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a53a6-bf60-4af3-a9e4-4278da43011a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:46.690336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-166a53a6-bf60-4af3-a9e4-4278da43011a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:46.709497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 872/1740] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:46.712219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea125c1-b3ce-41cc-8096-d5afd68dfc92 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:46.731665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:46.756555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea125c1-b3ce-41cc-8096-d5afd68dfc92 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:46.772495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485bf63-659d-4c9c-b2df-9344afdf4f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:46.773616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485bf63-659d-4c9c-b2df-9344afdf4f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.777642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485bf63-659d-4c9c-b2df-9344afdf4f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.778908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485bf63-659d-4c9c-b2df-9344afdf4f91 admin admin] PATH_INFO: `/v3/projects/e9579225e79b44babbfdba246b229acb/users/258361576e804fb596e22af3d12e7856/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.788479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485bf63-659d-4c9c-b2df-9344afdf4f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9579225e79b44babbfdba246b229acb, user_id=258361576e804fb596e22af3d12e7856, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:46.793866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485bf63-659d-4c9c-b2df-9344afdf4f91 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:46.850300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 873/1741] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:46 2022] PUT /identity/v3/projects/e9579225e79b44babbfdba246b229acb/users/258361576e804fb596e22af3d12e7856/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:46.850300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95392431-e428-42c0-9117-3cbe0bdb3516 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:46.850300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95392431-e428-42c0-9117-3cbe0bdb3516 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:46.850300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95392431-e428-42c0-9117-3cbe0bdb3516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:46.894440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-95392431-e428-42c0-9117-3cbe0bdb3516 None None] MFA Rules not processed for user `a7e574dc676841dea1f23a06c333ff14`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:46.898848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-95392431-e428-42c0-9117-3cbe0bdb3516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:46.940837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166a53a6-bf60-4af3-a9e4-4278da43011a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:47.067913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 874/1742] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:47.069663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013b11-661f-4f4d-9030-02e132b04f63 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:47.069949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013b11-661f-4f4d-9030-02e132b04f63 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:47.071006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3013b11-661f-4f4d-9030-02e132b04f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:47.113426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b634b7-23de-4cb0-a5ac-6e9082bf7797 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:47.114868 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b634b7-23de-4cb0-a5ac-6e9082bf7797 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:47.115340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b634b7-23de-4cb0-a5ac-6e9082bf7797 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:47.115784 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b634b7-23de-4cb0-a5ac-6e9082bf7797 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:47.116340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e3013b11-661f-4f4d-9030-02e132b04f63 None None] MFA Rules not processed for user `258361576e804fb596e22af3d12e7856`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:47.117695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e3013b11-661f-4f4d-9030-02e132b04f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:47.138576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b634b7-23de-4cb0-a5ac-6e9082bf7797 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:47.141524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b634b7-23de-4cb0-a5ac-6e9082bf7797 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:47.162632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 869/1743] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:47.164656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472e05-2e12-4d4f-a8ba-9ce787e8453c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:47.165378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472e05-2e12-4d4f-a8ba-9ce787e8453c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:47.165981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25472e05-2e12-4d4f-a8ba-9ce787e8453c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:47.219675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-25472e05-2e12-4d4f-a8ba-9ce787e8453c None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:47.224307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-25472e05-2e12-4d4f-a8ba-9ce787e8453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:47.227057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:47.335401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 870/1744] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:47.350660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 875/1745] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:47.363830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472e05-2e12-4d4f-a8ba-9ce787e8453c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:47.367188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-25472e05-2e12-4d4f-a8ba-9ce787e8453c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:47.451720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:47.498284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25472e05-2e12-4d4f-a8ba-9ce787e8453c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:47.503025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297cfff6-ca39-4a23-b8fe-61131b5af45d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:47.503907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cfff6-ca39-4a23-b8fe-61131b5af45d service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:47.504190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cfff6-ca39-4a23-b8fe-61131b5af45d service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:47.504465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297cfff6-ca39-4a23-b8fe-61131b5af45d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:47.562807 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3013b11-661f-4f4d-9030-02e132b04f63 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:47.565086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e3013b11-661f-4f4d-9030-02e132b04f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:47.604817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3013b11-661f-4f4d-9030-02e132b04f63 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:47.626623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c93fe-11ad-4c23-96d0-6850f458de2a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:47.627464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c93fe-11ad-4c23-96d0-6850f458de2a service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:47.627857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c93fe-11ad-4c23-96d0-6850f458de2a service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:47.628318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c93fe-11ad-4c23-96d0-6850f458de2a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:47.659872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c93fe-11ad-4c23-96d0-6850f458de2a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:47.661788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865c93fe-11ad-4c23-96d0-6850f458de2a service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:47.674881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cfff6-ca39-4a23-b8fe-61131b5af45d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:47.676564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297cfff6-ca39-4a23-b8fe-61131b5af45d service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:47.687588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 871/1746] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:47 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:47.689207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 876/1747] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:13:47.707035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0910bb-8ed9-426d-96b9-67667ae736cd None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:47.707384 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0910bb-8ed9-426d-96b9-67667ae736cd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:47.707610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0910bb-8ed9-426d-96b9-67667ae736cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:47.735292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-5f0910bb-8ed9-426d-96b9-67667ae736cd None None] MFA Rules not processed for user `ff944e4cf29b4297acc63b591d90daaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:47.736743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5f0910bb-8ed9-426d-96b9-67667ae736cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:47.839989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 872/1748] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:48.782033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11692e32-4dcc-486e-88ea-acf6cf1906f1 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:48.782565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11692e32-4dcc-486e-88ea-acf6cf1906f1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:48.782565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11692e32-4dcc-486e-88ea-acf6cf1906f1 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:48.783998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 877/1749] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:48.825050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605672b9-389c-4436-a879-bae72636d1e1 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:48.825599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605672b9-389c-4436-a879-bae72636d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:48.825599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605672b9-389c-4436-a879-bae72636d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:48.863229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-605672b9-389c-4436-a879-bae72636d1e1 None None] MFA Rules not processed for user `6def7fd117dc4b7598a1864e63f09427`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:48.865000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-605672b9-389c-4436-a879-bae72636d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:48.906225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 873/1750] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:13:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:48.907447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11692e32-4dcc-486e-88ea-acf6cf1906f1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:48.909232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-11692e32-4dcc-486e-88ea-acf6cf1906f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:48.925336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605672b9-389c-4436-a879-bae72636d1e1 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:48.938692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605672b9-389c-4436-a879-bae72636d1e1 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:48.947683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:48.948529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:48.948974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:48.949334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:48.984044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:48.986091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:48.999360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 874/1751] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:49.054316 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11692e32-4dcc-486e-88ea-acf6cf1906f1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:49.068752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:49.069877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:49.070474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:49.070757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] PATH_INFO: `/v3/users/acdd9c8b45a34e61b4c1642966862a7b` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:49.072642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=acdd9c8b45a34e61b4c1642966862a7b)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:49.074606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:49.200353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acdd9c8b45a34e61b4c1642966862a7b'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.218386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acdd9c8b45a34e61b4c1642966862a7b'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.219407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acdd9c8b45a34e61b4c1642966862a7b'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.239183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acdd9c8b45a34e61b4c1642966862a7b'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.240060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13d063ca-4ee5-4eae-b4f3-d03197df75ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acdd9c8b45a34e61b4c1642966862a7b'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.295745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 878/1752] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:48 2022] DELETE /identity/v3/users/acdd9c8b45a34e61b4c1642966862a7b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:49.329687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:49.331562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:49.372210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a5761-ac19-4b97-b863-e85a22e90173 service placement] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:49.512168 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa644ceb-2731-4f5b-907b-8cb2ba590282 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:49.512468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa644ceb-2731-4f5b-907b-8cb2ba590282 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:49.512913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa644ceb-2731-4f5b-907b-8cb2ba590282 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:49.514862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 879/1753] 10.222.0.69 () {54 vars in 912 bytes} [Fri May 20 09:13:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:49.530905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeca1f6-80ad-4e8f-a377-2200251132b3 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:49.531109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeca1f6-80ad-4e8f-a377-2200251132b3 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:49.531433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eeca1f6-80ad-4e8f-a377-2200251132b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:49.576761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2eeca1f6-80ad-4e8f-a377-2200251132b3 None None] MFA Rules not processed for user `349e99511c7c4307b27dc90b01f2dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:49.580503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2eeca1f6-80ad-4e8f-a377-2200251132b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:49.625732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:49.626692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:49.626993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:49.627287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] PATH_INFO: `/v3/projects/06b573cd178a4483bd978ab79f920fed` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:49.630176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06b573cd178a4483bd978ab79f920fed)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:49.633809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:49.660084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:49.675798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:49.713583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 880/1754] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:13:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:49.736215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06b573cd178a4483bd978ab79f920fed', 'request_id': 'req-2c44e808-f913-4fec-8232-78892c2b2ff3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.741182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeca1f6-80ad-4e8f-a377-2200251132b3 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:49.742834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2eeca1f6-80ad-4e8f-a377-2200251132b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:49.750481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06b573cd178a4483bd978ab79f920fed', 'request_id': 'req-2c44e808-f913-4fec-8232-78892c2b2ff3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.758328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eeca1f6-80ad-4e8f-a377-2200251132b3 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:49.803308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06b573cd178a4483bd978ab79f920fed', 'request_id': 'req-2c44e808-f913-4fec-8232-78892c2b2ff3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.853341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:49.853802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] The token cache is being invalidate because project 06b573cd178a4483bd978ab79f920fed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:49.854153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:49.854243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:49.854485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:49.855196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.856768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:49.858976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:49.866254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 875/1755] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:49 2022] DELETE /identity/v3/projects/06b573cd178a4483bd978ab79f920fed => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:49.873387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:49.874338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:49.875302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:49.875968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:49.888433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 881/1756] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:49.889381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:49.890827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:50.003331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf3aaa-e810-4b25-9a97-f79e98f9fbc9 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:50.008534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c44e808-f913-4fec-8232-78892c2b2ff3 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:50.014549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:50.015540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:50.015855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:50.016736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] PATH_INFO: `/v3/projects/c951c8772bd641f9b9ffff5eec22d6a3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:50.017531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:50.018475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:50.018759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:50.018857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c951c8772bd641f9b9ffff5eec22d6a3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:50.019232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] PATH_INFO: `/v3/users/624bb13c2d094dc7820b3304ff0cc470` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:50.020527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624bb13c2d094dc7820b3304ff0cc470)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:50.020927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:50.023998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:50.123945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c951c8772bd641f9b9ffff5eec22d6a3', 'request_id': 'req-7bda79c5-4232-434c-be0f-83e4b4731722'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.131148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c951c8772bd641f9b9ffff5eec22d6a3', 'request_id': 'req-7bda79c5-4232-434c-be0f-83e4b4731722'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.139499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c951c8772bd641f9b9ffff5eec22d6a3', 'request_id': 'req-7bda79c5-4232-434c-be0f-83e4b4731722'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.164814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] The token cache is being invalidate because project c951c8772bd641f9b9ffff5eec22d6a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:50.165507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.166453 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624bb13c2d094dc7820b3304ff0cc470'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.167749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.169449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624bb13c2d094dc7820b3304ff0cc470'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.170133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 882/1757] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:49 2022] DELETE /identity/v3/projects/c951c8772bd641f9b9ffff5eec22d6a3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:50.171188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624bb13c2d094dc7820b3304ff0cc470'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.174155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:50.175868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:50.196512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624bb13c2d094dc7820b3304ff0cc470'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.205465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624bb13c2d094dc7820b3304ff0cc470'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.217285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:50.240634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 876/1758] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:49 2022] DELETE /identity/v3/users/624bb13c2d094dc7820b3304ff0cc470 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:50.297804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bda79c5-4232-434c-be0f-83e4b4731722 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:50.383158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:50.385656 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:50.386038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:50.386625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] PATH_INFO: `/v3/users/c091b532f31c40b990a8333d94e0ecb0` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:50.392857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c091b532f31c40b990a8333d94e0ecb0)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:50.403123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:50.527973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c091b532f31c40b990a8333d94e0ecb0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.543072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c091b532f31c40b990a8333d94e0ecb0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.544170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c091b532f31c40b990a8333d94e0ecb0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.555887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c091b532f31c40b990a8333d94e0ecb0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.556800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-92ff574f-b8f2-41c9-9a52-b138b36c3e74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c091b532f31c40b990a8333d94e0ecb0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:50.598208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 883/1759] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:50 2022] DELETE /identity/v3/users/c091b532f31c40b990a8333d94e0ecb0 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:13:50.987294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:50.989663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:51.017513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e518fe-9c3b-43ca-85e0-7e172a2e27c7 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.055959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:51.118422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.119809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.120354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.120793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] PATH_INFO: `/v3/projects/c72c4fd7f4894234b985a06c51bcf361` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.124364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c72c4fd7f4894234b985a06c51bcf361)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.129665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.224492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c72c4fd7f4894234b985a06c51bcf361', 'request_id': 'req-1043a1c5-5acd-4a51-8b7d-019dc89cd061'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.236394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c72c4fd7f4894234b985a06c51bcf361', 'request_id': 'req-1043a1c5-5acd-4a51-8b7d-019dc89cd061'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.254406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c72c4fd7f4894234b985a06c51bcf361', 'request_id': 'req-1043a1c5-5acd-4a51-8b7d-019dc89cd061'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.285069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] The token cache is being invalidate because project c72c4fd7f4894234b985a06c51bcf361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:51.285374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.287087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.289640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 877/1760] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:50 2022] DELETE /identity/v3/projects/c72c4fd7f4894234b985a06c51bcf361 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:51.296657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c0f6-bc29-4bb6-8f87-adf7a97d97b3 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.297752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c0f6-bc29-4bb6-8f87-adf7a97d97b3 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.297970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a692c0f6-bc29-4bb6-8f87-adf7a97d97b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.316160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.318422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:51.324406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a692c0f6-bc29-4bb6-8f87-adf7a97d97b3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:51.325748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a692c0f6-bc29-4bb6-8f87-adf7a97d97b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:51.389842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:51.414006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1043a1c5-5acd-4a51-8b7d-019dc89cd061 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.420711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.422309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.422712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.423082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] PATH_INFO: `/v3/projects/3cdc9d7c04474a868c926cf065427f5e` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.425651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cdc9d7c04474a868c926cf065427f5e)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.428076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.436294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 884/1761] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:51.439064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72430ee1-a907-4f4a-b68f-81182e7c680a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.439383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72430ee1-a907-4f4a-b68f-81182e7c680a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.439668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72430ee1-a907-4f4a-b68f-81182e7c680a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.480590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-72430ee1-a907-4f4a-b68f-81182e7c680a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:51.482182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-72430ee1-a907-4f4a-b68f-81182e7c680a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:51.488895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cdc9d7c04474a868c926cf065427f5e', 'request_id': 'req-10377738-5443-4731-8562-0e805dae6b07'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.494256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:51.504782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cdc9d7c04474a868c926cf065427f5e', 'request_id': 'req-10377738-5443-4731-8562-0e805dae6b07'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.511874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cdc9d7c04474a868c926cf065427f5e', 'request_id': 'req-10377738-5443-4731-8562-0e805dae6b07'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.530945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] The token cache is being invalidate because project 3cdc9d7c04474a868c926cf065427f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:51.531246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.532046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-10377738-5443-4731-8562-0e805dae6b07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:51.534717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 878/1762] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:51 2022] DELETE /identity/v3/projects/3cdc9d7c04474a868c926cf065427f5e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:51.547334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5919b-abaf-4db7-bbe3-f878604fc852 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.547632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5919b-abaf-4db7-bbe3-f878604fc852 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.547960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc5919b-abaf-4db7-bbe3-f878604fc852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.568588 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0bc5919b-abaf-4db7-bbe3-f878604fc852 None None] MFA Rules not processed for user `bfaaa367491745708334c5f35331bf76`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:51.569804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0bc5919b-abaf-4db7-bbe3-f878604fc852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:51.575991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 885/1763] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:51.577317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7add37-6439-4615-975a-2e90cf6ddd36 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.577606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7add37-6439-4615-975a-2e90cf6ddd36 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.577882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7add37-6439-4615-975a-2e90cf6ddd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.599082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-ca7add37-6439-4615-975a-2e90cf6ddd36 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:51.600568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ca7add37-6439-4615-975a-2e90cf6ddd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:51.609793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 886/1764] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:51.610573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7add37-6439-4615-975a-2e90cf6ddd36 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.622184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7add37-6439-4615-975a-2e90cf6ddd36 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.626898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.627583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.627860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.628120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.628848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:51.629152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.630752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.639501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 879/1765] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:51.641278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c48710-4fd0-48ce-8fc0-f2a7a79794e2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.641741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c48710-4fd0-48ce-8fc0-f2a7a79794e2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.642274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c48710-4fd0-48ce-8fc0-f2a7a79794e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.642988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 887/1766] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:51.649228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.661314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf5a29e-8564-4c61-956c-8adf566f7d3f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.665885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235b79b-ee61-4222-9895-6a93504de7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.666248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-59c48710-4fd0-48ce-8fc0-f2a7a79794e2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:51.666756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b79b-ee61-4222-9895-6a93504de7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.666862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b79b-ee61-4222-9895-6a93504de7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.667075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235b79b-ee61-4222-9895-6a93504de7e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.667599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-59c48710-4fd0-48ce-8fc0-f2a7a79794e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:51.667947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235b79b-ee61-4222-9895-6a93504de7e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.669906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235b79b-ee61-4222-9895-6a93504de7e7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.680993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 880/1767] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:51.681320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c48710-4fd0-48ce-8fc0-f2a7a79794e2 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.691458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c48710-4fd0-48ce-8fc0-f2a7a79794e2 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.696865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.697687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.698078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.698537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.699383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:51.699776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.701800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.707456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 888/1768] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:13:51.711234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235b79b-ee61-4222-9895-6a93504de7e7 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.718239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 881/1769] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:13:51.719813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.731200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec1f1fe-a020-4715-830b-28aef7f8b07a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.796117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5603-aa15-4a1e-a036-13f4524ab74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.796424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235b79b-ee61-4222-9895-6a93504de7e7 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.796851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5603-aa15-4a1e-a036-13f4524ab74c admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.797131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5603-aa15-4a1e-a036-13f4524ab74c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.797416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc5603-aa15-4a1e-a036-13f4524ab74c admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.798344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc5603-aa15-4a1e-a036-13f4524ab74c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.800213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc5603-aa15-4a1e-a036-13f4524ab74c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.801997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a0654-51a5-43b6-af6d-8a1c3446869c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.802802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a0654-51a5-43b6-af6d-8a1c3446869c admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.803099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a0654-51a5-43b6-af6d-8a1c3446869c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.803401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a0654-51a5-43b6-af6d-8a1c3446869c admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.804168 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a0654-51a5-43b6-af6d-8a1c3446869c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.807035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a0654-51a5-43b6-af6d-8a1c3446869c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.834711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 889/1770] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:51.839347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 882/1771] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:51.839472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a0654-51a5-43b6-af6d-8a1c3446869c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.840291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5603-aa15-4a1e-a036-13f4524ab74c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.849509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a0654-51a5-43b6-af6d-8a1c3446869c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.890872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:51.923618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee677-7cb8-4541-ba9a-cd85a456b815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.924546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee677-7cb8-4541-ba9a-cd85a456b815 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.925048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee677-7cb8-4541-ba9a-cd85a456b815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.925526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee677-7cb8-4541-ba9a-cd85a456b815 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.926363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeee677-7cb8-4541-ba9a-cd85a456b815 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.928278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeee677-7cb8-4541-ba9a-cd85a456b815 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.943389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 890/1772] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:51.944930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee677-7cb8-4541-ba9a-cd85a456b815 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:51.947066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:51.959024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee677-7cb8-4541-ba9a-cd85a456b815 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.961681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc5603-aa15-4a1e-a036-13f4524ab74c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:51.966692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d1c55-52ed-421f-897f-e8b80d13d361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.967965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d1c55-52ed-421f-897f-e8b80d13d361 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.968356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d1c55-52ed-421f-897f-e8b80d13d361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.970319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d1c55-52ed-421f-897f-e8b80d13d361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.970818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d1c55-52ed-421f-897f-e8b80d13d361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.973702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824932b-a207-47fb-90fb-8d6011f64ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:51.974630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824932b-a207-47fb-90fb-8d6011f64ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:51.975370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d1c55-52ed-421f-897f-e8b80d13d361 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.975774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824932b-a207-47fb-90fb-8d6011f64ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:51.977167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8824932b-a207-47fb-90fb-8d6011f64ca1 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:51.977167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824932b-a207-47fb-90fb-8d6011f64ca1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:51.982469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8824932b-a207-47fb-90fb-8d6011f64ca1 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:51.993104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 891/1773] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:51.997373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d1c55-52ed-421f-897f-e8b80d13d361 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.009512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d1c55-52ed-421f-897f-e8b80d13d361 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.010023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 883/1774] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:51 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:13:52.014101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824932b-a207-47fb-90fb-8d6011f64ca1 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.016127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f910a-435c-4d8b-bc33-14ddcad950f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.017094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f910a-435c-4d8b-bc33-14ddcad950f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.017498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f910a-435c-4d8b-bc33-14ddcad950f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.017808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f910a-435c-4d8b-bc33-14ddcad950f3 admin admin] PATH_INFO: `/v3/projects/c5c4ca3bacad4a40ba9aa63313d75417/users/209ab22afc2d4e908f3b7d1443e656fb/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.026280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8824932b-a207-47fb-90fb-8d6011f64ca1 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.032266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b4127-fdea-4100-83ae-55aede516cef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.033002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b4127-fdea-4100-83ae-55aede516cef admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.033290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b4127-fdea-4100-83ae-55aede516cef admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.033617 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5b4127-fdea-4100-83ae-55aede516cef admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.034680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b4127-fdea-4100-83ae-55aede516cef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.037224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5b4127-fdea-4100-83ae-55aede516cef admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.038322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f910a-435c-4d8b-bc33-14ddcad950f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c4ca3bacad4a40ba9aa63313d75417, user_id=209ab22afc2d4e908f3b7d1443e656fb, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.041174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f910a-435c-4d8b-bc33-14ddcad950f3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.049878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 884/1775] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:52.054030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b4127-fdea-4100-83ae-55aede516cef admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.061987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 892/1776] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:51 2022] PUT /identity/v3/projects/c5c4ca3bacad4a40ba9aa63313d75417/users/209ab22afc2d4e908f3b7d1443e656fb/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:52.065216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5b4127-fdea-4100-83ae-55aede516cef admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.072989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7109af8-04cd-4259-a91e-fb1ff874bdf9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.073497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7109af8-04cd-4259-a91e-fb1ff874bdf9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.073784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7109af8-04cd-4259-a91e-fb1ff874bdf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.101297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d7109af8-04cd-4259-a91e-fb1ff874bdf9 None None] MFA Rules not processed for user `209ab22afc2d4e908f3b7d1443e656fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:52.102527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d7109af8-04cd-4259-a91e-fb1ff874bdf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:52.141222 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234cfa3-5002-4993-a504-681cfefbb24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.142231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234cfa3-5002-4993-a504-681cfefbb24a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.142875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234cfa3-5002-4993-a504-681cfefbb24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.143348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5234cfa3-5002-4993-a504-681cfefbb24a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.144436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234cfa3-5002-4993-a504-681cfefbb24a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.146262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5234cfa3-5002-4993-a504-681cfefbb24a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.159644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 885/1777] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:13:52.169380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234cfa3-5002-4993-a504-681cfefbb24a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.188073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5234cfa3-5002-4993-a504-681cfefbb24a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.195713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 893/1778] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:52.198288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61775571-cd10-49a9-8b6e-c3ed8fef144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.199206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61775571-cd10-49a9-8b6e-c3ed8fef144a admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.199521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61775571-cd10-49a9-8b6e-c3ed8fef144a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.199632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7109af8-04cd-4259-a91e-fb1ff874bdf9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.199834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61775571-cd10-49a9-8b6e-c3ed8fef144a admin admin] PATH_INFO: `/v3/projects/28af0d79a6a04140a2aba2537b6d710f/users/9f9830253bcf4d108b55d2f53470e3ad/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.215687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7109af8-04cd-4259-a91e-fb1ff874bdf9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.223123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61775571-cd10-49a9-8b6e-c3ed8fef144a admin admin] RBAC: Authorizing `identity:create_grant(project_id=28af0d79a6a04140a2aba2537b6d710f, user_id=9f9830253bcf4d108b55d2f53470e3ad, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.223819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112288e-9191-4fe6-9542-d6303e6d03ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.225392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112288e-9191-4fe6-9542-d6303e6d03ff admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.225693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112288e-9191-4fe6-9542-d6303e6d03ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.225972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112288e-9191-4fe6-9542-d6303e6d03ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.226312 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61775571-cd10-49a9-8b6e-c3ed8fef144a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.226681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112288e-9191-4fe6-9542-d6303e6d03ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.230485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112288e-9191-4fe6-9542-d6303e6d03ff admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.251422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 886/1779] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:52 2022] PUT /identity/v3/projects/28af0d79a6a04140a2aba2537b6d710f/users/9f9830253bcf4d108b55d2f53470e3ad/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:52.256517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f70e7-9396-4b32-88c0-eac877bb2e45 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.256913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f70e7-9396-4b32-88c0-eac877bb2e45 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.257508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f70e7-9396-4b32-88c0-eac877bb2e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.275812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 894/1780] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:52 2022] POST /identity/v3/projects => generated 373 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:52.276680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112288e-9191-4fe6-9542-d6303e6d03ff admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.302945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9a5f70e7-9396-4b32-88c0-eac877bb2e45 None None] MFA Rules not processed for user `9f9830253bcf4d108b55d2f53470e3ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:52.304436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9a5f70e7-9396-4b32-88c0-eac877bb2e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:52.368904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112288e-9191-4fe6-9542-d6303e6d03ff admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.376188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbe7bf-806f-4914-a424-c026eaaf84dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.376877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbe7bf-806f-4914-a424-c026eaaf84dc admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.377161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbe7bf-806f-4914-a424-c026eaaf84dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.377438 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fbe7bf-806f-4914-a424-c026eaaf84dc admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.378246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbe7bf-806f-4914-a424-c026eaaf84dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.380241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 887/1781] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:52.381710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fbe7bf-806f-4914-a424-c026eaaf84dc admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.384773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f70e7-9396-4b32-88c0-eac877bb2e45 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.391231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:52.413540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f70e7-9396-4b32-88c0-eac877bb2e45 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.419029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 895/1782] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:52 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:52.419504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbe7bf-806f-4914-a424-c026eaaf84dc admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.420025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23514e75-9203-41a4-87a5-59b8fae939db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.421074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23514e75-9203-41a4-87a5-59b8fae939db admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.421496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23514e75-9203-41a4-87a5-59b8fae939db admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.421877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23514e75-9203-41a4-87a5-59b8fae939db admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.423341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23514e75-9203-41a4-87a5-59b8fae939db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.426672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23514e75-9203-41a4-87a5-59b8fae939db admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.446794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fbe7bf-806f-4914-a424-c026eaaf84dc admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.453599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3b17e-8318-415e-bab4-b96051da579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.454272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3b17e-8318-415e-bab4-b96051da579e admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.454474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3b17e-8318-415e-bab4-b96051da579e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.454755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3b17e-8318-415e-bab4-b96051da579e admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.455739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3b17e-8318-415e-bab4-b96051da579e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.457653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b3b17e-8318-415e-bab4-b96051da579e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.471299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 896/1783] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:52.475848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3b17e-8318-415e-bab4-b96051da579e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.491299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 888/1784] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:52 2022] POST /identity/v3/projects => generated 385 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:52.494133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23514e75-9203-41a4-87a5-59b8fae939db admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.495987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3b17e-8318-415e-bab4-b96051da579e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.521202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:52.574900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2213-5ba6-413f-aaec-7884d3969f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.576181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2213-5ba6-413f-aaec-7884d3969f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.576470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2213-5ba6-413f-aaec-7884d3969f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.576890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965d2213-5ba6-413f-aaec-7884d3969f33 admin admin] PATH_INFO: `/v3/projects/9c1d110ff7734fb49aa31a6ad96b69e7/users/f2cca9e348f44dcbb30ea72d98d2b34e/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.596106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23514e75-9203-41a4-87a5-59b8fae939db admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.601263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2213-5ba6-413f-aaec-7884d3969f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1d110ff7734fb49aa31a6ad96b69e7, user_id=f2cca9e348f44dcbb30ea72d98d2b34e, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.601660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41fcc7-be61-4d84-8358-e83c9fe5b144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.602461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41fcc7-be61-4d84-8358-e83c9fe5b144 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.602807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41fcc7-be61-4d84-8358-e83c9fe5b144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.603153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41fcc7-be61-4d84-8358-e83c9fe5b144 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.603984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41fcc7-be61-4d84-8358-e83c9fe5b144 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.604825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965d2213-5ba6-413f-aaec-7884d3969f33 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.606141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41fcc7-be61-4d84-8358-e83c9fe5b144 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.620025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 897/1785] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:52 2022] PUT /identity/v3/projects/9c1d110ff7734fb49aa31a6ad96b69e7/users/f2cca9e348f44dcbb30ea72d98d2b34e/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:13:52.624253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2213-5ba6-413f-aaec-7884d3969f33 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.628708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 889/1786] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:52 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:52.632751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41fcc7-be61-4d84-8358-e83c9fe5b144 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.644094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41fcc7-be61-4d84-8358-e83c9fe5b144 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.715941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965d2213-5ba6-413f-aaec-7884d3969f33 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.737837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63d50d-e3f5-4e05-ab1b-6a47650c58ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.740557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d50d-e3f5-4e05-ab1b-6a47650c58ba admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.741291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d50d-e3f5-4e05-ab1b-6a47650c58ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.741599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d50d-e3f5-4e05-ab1b-6a47650c58ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.742624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63d50d-e3f5-4e05-ab1b-6a47650c58ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.744891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63d50d-e3f5-4e05-ab1b-6a47650c58ba admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.749509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6db2ca-db83-4f64-bdb8-a8d4cac189ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.751094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6db2ca-db83-4f64-bdb8-a8d4cac189ef admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.753731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6db2ca-db83-4f64-bdb8-a8d4cac189ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.754043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6db2ca-db83-4f64-bdb8-a8d4cac189ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.755272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6db2ca-db83-4f64-bdb8-a8d4cac189ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.761916 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6db2ca-db83-4f64-bdb8-a8d4cac189ef admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.784373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 898/1787] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:13:52.788575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63d50d-e3f5-4e05-ab1b-6a47650c58ba admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.795444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 890/1788] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:52.799444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6db2ca-db83-4f64-bdb8-a8d4cac189ef admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:52.808101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63d50d-e3f5-4e05-ab1b-6a47650c58ba admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.824773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5d25f0-d353-44e9-a300-f492e663cf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.826833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6db2ca-db83-4f64-bdb8-a8d4cac189ef admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:52.828963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d25f0-d353-44e9-a300-f492e663cf13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.829434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d25f0-d353-44e9-a300-f492e663cf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.829805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5d25f0-d353-44e9-a300-f492e663cf13 admin admin] PATH_INFO: `/v3/projects/9c1d110ff7734fb49aa31a6ad96b69e7/users/f2cca9e348f44dcbb30ea72d98d2b34e/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.834476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d25f0-d353-44e9-a300-f492e663cf13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1d110ff7734fb49aa31a6ad96b69e7, user_id=f2cca9e348f44dcbb30ea72d98d2b34e, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.837609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5d25f0-d353-44e9-a300-f492e663cf13 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.838259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8baf7b-8c88-4180-be69-283e0cdf6119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:52.842085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8baf7b-8c88-4180-be69-283e0cdf6119 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.843089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8baf7b-8c88-4180-be69-283e0cdf6119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.844829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8baf7b-8c88-4180-be69-283e0cdf6119 admin admin] PATH_INFO: `/v3/projects/4cfa4b5d1de34a6db71c61e0bff3041c/users/ed9da8d86946472ab326c9845f225b6f/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.852943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 899/1789] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:52 2022] PUT /identity/v3/projects/9c1d110ff7734fb49aa31a6ad96b69e7/users/f2cca9e348f44dcbb30ea72d98d2b34e/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:52.873339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd35347-5d82-42c9-8afd-957235c6a1e4 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:52.874427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd35347-5d82-42c9-8afd-957235c6a1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:52.874844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd35347-5d82-42c9-8afd-957235c6a1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:52.892879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8baf7b-8c88-4180-be69-283e0cdf6119 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cfa4b5d1de34a6db71c61e0bff3041c, user_id=ed9da8d86946472ab326c9845f225b6f, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:52.901993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8baf7b-8c88-4180-be69-283e0cdf6119 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:52.938901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-ccd35347-5d82-42c9-8afd-957235c6a1e4 None None] MFA Rules not processed for user `f2cca9e348f44dcbb30ea72d98d2b34e`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:52.941511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ccd35347-5d82-42c9-8afd-957235c6a1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:52.944125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 891/1790] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:52 2022] PUT /identity/v3/projects/4cfa4b5d1de34a6db71c61e0bff3041c/users/ed9da8d86946472ab326c9845f225b6f/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:52.950286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8baf7b-8c88-4180-be69-283e0cdf6119 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:53.008500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:53.075063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 900/1791] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:53.089293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd35347-5d82-42c9-8afd-957235c6a1e4 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:53.093505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ccd35347-5d82-42c9-8afd-957235c6a1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:53.146505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8baf7b-8c88-4180-be69-283e0cdf6119 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:53.161018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:53.161997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:53.162343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:53.163647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:53.164757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:53.166622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:53.200804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 892/1792] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:53.201862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:53.207182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:53.221867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:53.243923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd35347-5d82-42c9-8afd-957235c6a1e4 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:53.250320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6af6e-0463-4709-9132-82de7bda1333 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:53.251296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6af6e-0463-4709-9132-82de7bda1333 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:53.251675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6af6e-0463-4709-9132-82de7bda1333 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:53.252067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6af6e-0463-4709-9132-82de7bda1333 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:53.254370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f4c6af6e-0463-4709-9132-82de7bda1333 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:53.307077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0c149-3847-4823-bd4d-8653cd35c838 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:53.311991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58177d11-852e-4f5f-9cd7-a9593b98f388 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:53.313186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58177d11-852e-4f5f-9cd7-a9593b98f388 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:53.313949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58177d11-852e-4f5f-9cd7-a9593b98f388 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:53.314429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58177d11-852e-4f5f-9cd7-a9593b98f388 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:53.340434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6af6e-0463-4709-9132-82de7bda1333 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:53.343173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58177d11-852e-4f5f-9cd7-a9593b98f388 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:53.343915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c6af6e-0463-4709-9132-82de7bda1333 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:53.345455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58177d11-852e-4f5f-9cd7-a9593b98f388 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:53.358227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 893/1793] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:53 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:53.359119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58177d11-852e-4f5f-9cd7-a9593b98f388 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:53.369974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 901/1794] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:53 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:53.384106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58177d11-852e-4f5f-9cd7-a9593b98f388 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:53.399936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c453-f748-4954-915d-9476709cb38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:53.401451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c453-f748-4954-915d-9476709cb38f admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:53.401451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c453-f748-4954-915d-9476709cb38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:53.401451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781c453-f748-4954-915d-9476709cb38f admin admin] PATH_INFO: `/v3/projects/4cfa4b5d1de34a6db71c61e0bff3041c/users/ed9da8d86946472ab326c9845f225b6f/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:53.409038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781c453-f748-4954-915d-9476709cb38f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cfa4b5d1de34a6db71c61e0bff3041c, user_id=ed9da8d86946472ab326c9845f225b6f, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:53.412193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781c453-f748-4954-915d-9476709cb38f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:53.459526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 894/1795] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:53 2022] PUT /identity/v3/projects/4cfa4b5d1de34a6db71c61e0bff3041c/users/ed9da8d86946472ab326c9845f225b6f/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:53.461126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae49a42-c871-4dc1-965c-2fcceb739fc1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:53.461505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae49a42-c871-4dc1-965c-2fcceb739fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:53.461848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae49a42-c871-4dc1-965c-2fcceb739fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:53.530194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c453-f748-4954-915d-9476709cb38f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:53.533827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3ae49a42-c871-4dc1-965c-2fcceb739fc1 None None] MFA Rules not processed for user `ed9da8d86946472ab326c9845f225b6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:53.536121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3ae49a42-c871-4dc1-965c-2fcceb739fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:53.613624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781c453-f748-4954-915d-9476709cb38f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:53.650388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73455fda-adf3-4a15-98cd-fb2e66805ffa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:53.651091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73455fda-adf3-4a15-98cd-fb2e66805ffa service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:53.651365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73455fda-adf3-4a15-98cd-fb2e66805ffa service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:53.667286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73455fda-adf3-4a15-98cd-fb2e66805ffa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:53.726637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 902/1796] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:53.789643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae49a42-c871-4dc1-965c-2fcceb739fc1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:53.802680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae49a42-c871-4dc1-965c-2fcceb739fc1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:53.829729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cdb8d-c849-4921-a7e6-9e6a347d6738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:53.830372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cdb8d-c849-4921-a7e6-9e6a347d6738 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:53.830621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cdb8d-c849-4921-a7e6-9e6a347d6738 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:53.830893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cdb8d-c849-4921-a7e6-9e6a347d6738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:53.877020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:53.888462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73455fda-adf3-4a15-98cd-fb2e66805ffa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:53.889847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cdb8d-c849-4921-a7e6-9e6a347d6738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:53.891478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7cdb8d-c849-4921-a7e6-9e6a347d6738 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:53.904325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73455fda-adf3-4a15-98cd-fb2e66805ffa service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:53.933622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 903/1797] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:13:53.948902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 895/1798] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:54.193909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cdb8d-c849-4921-a7e6-9e6a347d6738 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:54.362350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cdb8d-c849-4921-a7e6-9e6a347d6738 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:54.370415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:54.373014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:54.373561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:54.374044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] PATH_INFO: `/v3/users/9f9830253bcf4d108b55d2f53470e3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:54.382715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f9830253bcf4d108b55d2f53470e3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:54.385170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:54.467797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f9830253bcf4d108b55d2f53470e3ad'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:54.492222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f9830253bcf4d108b55d2f53470e3ad'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:54.495680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f9830253bcf4d108b55d2f53470e3ad'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:54.510499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f9830253bcf4d108b55d2f53470e3ad'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:54.512088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f9830253bcf4d108b55d2f53470e3ad'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:54.541449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 904/1799] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:54 2022] DELETE /identity/v3/users/9f9830253bcf4d108b55d2f53470e3ad => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:54.546162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef6629-dddb-4ed4-9c2b-89b195cc6f6c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:54.546637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef6629-dddb-4ed4-9c2b-89b195cc6f6c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:54.546728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fef6629-dddb-4ed4-9c2b-89b195cc6f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:54.585928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9fef6629-dddb-4ed4-9c2b-89b195cc6f6c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:54.587556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9fef6629-dddb-4ed4-9c2b-89b195cc6f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:54.663505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:54.681714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 896/1800] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:54.703488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:54.705124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:54.778878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5faab8-d974-4f3a-894b-e25c304cce96 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:54.783956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fa374-1a5e-49a2-bd70-74f2a081303d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:54.784631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fa374-1a5e-49a2-bd70-74f2a081303d service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:54.784912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fa374-1a5e-49a2-bd70-74f2a081303d service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:54.785186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02fa374-1a5e-49a2-bd70-74f2a081303d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:54.797292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fa374-1a5e-49a2-bd70-74f2a081303d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:54.798953 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02fa374-1a5e-49a2-bd70-74f2a081303d service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:54.809300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 905/1801] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:55.125923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef6629-dddb-4ed4-9c2b-89b195cc6f6c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:55.127752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9fef6629-dddb-4ed4-9c2b-89b195cc6f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:55.146712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fef6629-dddb-4ed4-9c2b-89b195cc6f6c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:55.158053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:55.159448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:55.159870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:55.160247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] PATH_INFO: `/v3/projects/28af0d79a6a04140a2aba2537b6d710f` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:55.161645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28af0d79a6a04140a2aba2537b6d710f)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:55.163826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:55.214203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28af0d79a6a04140a2aba2537b6d710f', 'request_id': 'req-c92db351-6033-4091-905e-0d1fee7f23f6'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.223747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28af0d79a6a04140a2aba2537b6d710f', 'request_id': 'req-c92db351-6033-4091-905e-0d1fee7f23f6'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.230056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28af0d79a6a04140a2aba2537b6d710f', 'request_id': 'req-c92db351-6033-4091-905e-0d1fee7f23f6'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.241713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] The token cache is being invalidate because project 28af0d79a6a04140a2aba2537b6d710f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:55.241998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.242649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c92db351-6033-4091-905e-0d1fee7f23f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.244238 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 897/1802] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:55 2022] DELETE /identity/v3/projects/28af0d79a6a04140a2aba2537b6d710f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:55.248331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fa374-1a5e-49a2-bd70-74f2a081303d service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:55.257822 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02fa374-1a5e-49a2-bd70-74f2a081303d service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:55.269446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:55.345627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:55.346521 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:55.346799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:55.347067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] PATH_INFO: `/v3/users/ed9da8d86946472ab326c9845f225b6f` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:55.348604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed9da8d86946472ab326c9845f225b6f)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:55.350887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:55.433299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9da8d86946472ab326c9845f225b6f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.442400 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9da8d86946472ab326c9845f225b6f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.443513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9da8d86946472ab326c9845f225b6f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.456761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9da8d86946472ab326c9845f225b6f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.457845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9da8d86946472ab326c9845f225b6f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:55.481521 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 906/1803] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:55 2022] DELETE /identity/v3/users/ed9da8d86946472ab326c9845f225b6f => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:13:55.914859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abaf21-a796-4f8c-a758-685861b6ea6d None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:55.915404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abaf21-a796-4f8c-a758-685861b6ea6d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:55.915683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abaf21-a796-4f8c-a758-685861b6ea6d None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:55.917398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 898/1804] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:13:55.922229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:55.932787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:55.934875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267c42-4290-4bef-874a-14923f669aa3 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:55.935230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267c42-4290-4bef-874a-14923f669aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:55.935581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48267c42-4290-4bef-874a-14923f669aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.005605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-48267c42-4290-4bef-874a-14923f669aa3 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:56.007252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-48267c42-4290-4bef-874a-14923f669aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:56.104561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2180c6-0f93-4cd1-9ebf-7e2c82b3bdbe admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:56.110102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:56.111100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.111468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.111747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] PATH_INFO: `/v3/projects/4cfa4b5d1de34a6db71c61e0bff3041c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.113052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cfa4b5d1de34a6db71c61e0bff3041c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:56.115010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:56.183691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 899/1805] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:13:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:56.239157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:56.259699 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cfa4b5d1de34a6db71c61e0bff3041c', 'request_id': 'req-225883ad-cd01-49f8-870d-dd863de0e10d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:56.270208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cfa4b5d1de34a6db71c61e0bff3041c', 'request_id': 'req-225883ad-cd01-49f8-870d-dd863de0e10d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:56.285008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cfa4b5d1de34a6db71c61e0bff3041c', 'request_id': 'req-225883ad-cd01-49f8-870d-dd863de0e10d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:56.301052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267c42-4290-4bef-874a-14923f669aa3 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:56.302529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-48267c42-4290-4bef-874a-14923f669aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:56.308455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] The token cache is being invalidate because project 4cfa4b5d1de34a6db71c61e0bff3041c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:56.308868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:56.309663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-225883ad-cd01-49f8-870d-dd863de0e10d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:56.311872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 907/1806] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:55 2022] DELETE /identity/v3/projects/4cfa4b5d1de34a6db71c61e0bff3041c => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:56.316566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb843f-b54c-4c18-a514-2a7692529c7c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.316566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb843f-b54c-4c18-a514-2a7692529c7c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.316789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48267c42-4290-4bef-874a-14923f669aa3 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:56.317097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb843f-b54c-4c18-a514-2a7692529c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.356824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6fbb843f-b54c-4c18-a514-2a7692529c7c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:56.359900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6fbb843f-b54c-4c18-a514-2a7692529c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:56.404857 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:56.405628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.405911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.406345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.409886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:56.420918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:56.426658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:56.430993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:56.453563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 908/1807] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:56.459620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 900/1808] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:13:56.461877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e612e7-6994-46ac-bab6-4037b55c140b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.462265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e612e7-6994-46ac-bab6-4037b55c140b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.462640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e612e7-6994-46ac-bab6-4037b55c140b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.482593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b2e612e7-6994-46ac-bab6-4037b55c140b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:56.484225 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b2e612e7-6994-46ac-bab6-4037b55c140b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:56.494248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 909/1809] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:56.498495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:56.509499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60311b6a-e312-4673-84f3-6f9992bfe6ca service nova] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:56.514075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:56.514732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.515009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.515304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.516039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:13:56.516343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:56.517945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:56.531090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 901/1810] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:13:56.535622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e612e7-6994-46ac-bab6-4037b55c140b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:56.546002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e612e7-6994-46ac-bab6-4037b55c140b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:56.551803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b46ef-a2f9-47f0-8dd9-963a0bbd6809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:56.554201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b46ef-a2f9-47f0-8dd9-963a0bbd6809 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.554550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b46ef-a2f9-47f0-8dd9-963a0bbd6809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.554889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0b46ef-a2f9-47f0-8dd9-963a0bbd6809 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.555695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b46ef-a2f9-47f0-8dd9-963a0bbd6809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:56.557831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0b46ef-a2f9-47f0-8dd9-963a0bbd6809 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:56.591519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 910/1811] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:56 2022] POST /identity/v3/projects => generated 389 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:56.596057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:56.606825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a93043-30b3-4895-908a-7ee76d4af56c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:56.664283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb99a33-bebf-4954-bdc8-ed483e1a5550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:56.664971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb99a33-bebf-4954-bdc8-ed483e1a5550 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.665255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb99a33-bebf-4954-bdc8-ed483e1a5550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.665529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb99a33-bebf-4954-bdc8-ed483e1a5550 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.666228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb99a33-bebf-4954-bdc8-ed483e1a5550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:56.668217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb99a33-bebf-4954-bdc8-ed483e1a5550 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:56.691216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 902/1812] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:56 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:56.697841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b46ef-a2f9-47f0-8dd9-963a0bbd6809 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:56.710226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0b46ef-a2f9-47f0-8dd9-963a0bbd6809 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:56.716098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3471c-5c3f-4f84-8b3e-cedbe14fc93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:56.716762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3471c-5c3f-4f84-8b3e-cedbe14fc93e admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.717038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3471c-5c3f-4f84-8b3e-cedbe14fc93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.717316 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f3471c-5c3f-4f84-8b3e-cedbe14fc93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.718047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3471c-5c3f-4f84-8b3e-cedbe14fc93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:56.719716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f3471c-5c3f-4f84-8b3e-cedbe14fc93e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:56.730233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 911/1813] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:56.734254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb99a33-bebf-4954-bdc8-ed483e1a5550 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:56.744564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb99a33-bebf-4954-bdc8-ed483e1a5550 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:56.749623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884a4b0-3223-4491-815b-f988d02f51ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:56.750285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884a4b0-3223-4491-815b-f988d02f51ef admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.750566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884a4b0-3223-4491-815b-f988d02f51ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.750835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884a4b0-3223-4491-815b-f988d02f51ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.751613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884a4b0-3223-4491-815b-f988d02f51ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:56.753224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884a4b0-3223-4491-815b-f988d02f51ef admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:56.764332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 903/1814] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:56.768788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3471c-5c3f-4f84-8b3e-cedbe14fc93e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:56.779710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f3471c-5c3f-4f84-8b3e-cedbe14fc93e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:56.784468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a31b6e-1a69-4677-8040-f7c69dc5c2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:56.785353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31b6e-1a69-4677-8040-f7c69dc5c2ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.785630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31b6e-1a69-4677-8040-f7c69dc5c2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.785937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a31b6e-1a69-4677-8040-f7c69dc5c2ee admin admin] PATH_INFO: `/v3/projects/968e62e3cd394794bc673ec6d336d5e8/users/dac09bade8234939bcd4434e76cb97ae/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.800933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a31b6e-1a69-4677-8040-f7c69dc5c2ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=968e62e3cd394794bc673ec6d336d5e8, user_id=dac09bade8234939bcd4434e76cb97ae, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:56.803617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a31b6e-1a69-4677-8040-f7c69dc5c2ee admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:56.819515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 912/1815] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:56 2022] PUT /identity/v3/projects/968e62e3cd394794bc673ec6d336d5e8/users/dac09bade8234939bcd4434e76cb97ae/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:56.830200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f45ab-5687-4acb-a910-be9a0d924eb4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:56.830485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f45ab-5687-4acb-a910-be9a0d924eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:56.830757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f45ab-5687-4acb-a910-be9a0d924eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:56.858685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-910f45ab-5687-4acb-a910-be9a0d924eb4 None None] MFA Rules not processed for user `dac09bade8234939bcd4434e76cb97ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:56.859932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-910f45ab-5687-4acb-a910-be9a0d924eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:56.927896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 904/1816] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:56.932592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a31b6e-1a69-4677-8040-f7c69dc5c2ee admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:56.995311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a31b6e-1a69-4677-8040-f7c69dc5c2ee admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:57.000616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b65e-eed5-4749-9488-8124309423ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:57.001276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b65e-eed5-4749-9488-8124309423ef admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.001562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b65e-eed5-4749-9488-8124309423ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.001840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1b65e-eed5-4749-9488-8124309423ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.002532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1b65e-eed5-4749-9488-8124309423ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:57.004543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1b65e-eed5-4749-9488-8124309423ef admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:57.008883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:57.032502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 913/1817] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:13:56 2022] POST /identity/v3/projects => generated 389 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:57.037309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f45ab-5687-4acb-a910-be9a0d924eb4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:57.049036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f45ab-5687-4acb-a910-be9a0d924eb4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:57.094570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:57.108698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11636a69-3322-44c5-ba4e-41eef26a37ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:57.109378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636a69-3322-44c5-ba4e-41eef26a37ed admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.109663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636a69-3322-44c5-ba4e-41eef26a37ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.109939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11636a69-3322-44c5-ba4e-41eef26a37ed admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.110640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11636a69-3322-44c5-ba4e-41eef26a37ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:57.112604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11636a69-3322-44c5-ba4e-41eef26a37ed admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:57.135899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 905/1818] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:13:57 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:13:57.140124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b65e-eed5-4749-9488-8124309423ef admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:57.150955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1b65e-eed5-4749-9488-8124309423ef admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:57.155929 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3860c52-50c0-4486-be20-a0ddad4a8b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:57.156665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3860c52-50c0-4486-be20-a0ddad4a8b59 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.157021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3860c52-50c0-4486-be20-a0ddad4a8b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.157326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3860c52-50c0-4486-be20-a0ddad4a8b59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.158116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3860c52-50c0-4486-be20-a0ddad4a8b59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:57.159815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3860c52-50c0-4486-be20-a0ddad4a8b59 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:57.170989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 914/1819] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:57.175277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11636a69-3322-44c5-ba4e-41eef26a37ed admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:57.186516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11636a69-3322-44c5-ba4e-41eef26a37ed admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:57.191724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7e8c-64c2-4b5f-8778-4a6ca74dfeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:57.192842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7e8c-64c2-4b5f-8778-4a6ca74dfeeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.193121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7e8c-64c2-4b5f-8778-4a6ca74dfeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.193400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe7e8c-64c2-4b5f-8778-4a6ca74dfeeb admin admin] PATH_INFO: `/v3/projects/b2928d0b95a341a38549b76c06d6aa81/users/30c3ba6d924e4bb4a824d14b3d729984/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.208781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe7e8c-64c2-4b5f-8778-4a6ca74dfeeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2928d0b95a341a38549b76c06d6aa81, user_id=30c3ba6d924e4bb4a824d14b3d729984, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:57.211509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe7e8c-64c2-4b5f-8778-4a6ca74dfeeb admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:57.228521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 906/1820] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:57 2022] PUT /identity/v3/projects/b2928d0b95a341a38549b76c06d6aa81/users/30c3ba6d924e4bb4a824d14b3d729984/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:57.232780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3860c52-50c0-4486-be20-a0ddad4a8b59 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:57.248940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3860c52-50c0-4486-be20-a0ddad4a8b59 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:57.317904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7374c8a4-ac32-4bf3-8c56-24425915aaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:57.318256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374c8a4-ac32-4bf3-8c56-24425915aaa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.318621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374c8a4-ac32-4bf3-8c56-24425915aaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.318967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7374c8a4-ac32-4bf3-8c56-24425915aaa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.319996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374c8a4-ac32-4bf3-8c56-24425915aaa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:57.322487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7374c8a4-ac32-4bf3-8c56-24425915aaa1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:57.334724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 915/1821] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:13:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:13:57.339097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7e8c-64c2-4b5f-8778-4a6ca74dfeeb admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:57.354468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe7e8c-64c2-4b5f-8778-4a6ca74dfeeb admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:57.360143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:57.361188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.361685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.362037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] PATH_INFO: `/v3/projects/b2928d0b95a341a38549b76c06d6aa81/users/30c3ba6d924e4bb4a824d14b3d729984/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.364237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2928d0b95a341a38549b76c06d6aa81, user_id=30c3ba6d924e4bb4a824d14b3d729984, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:57.367511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:57.382477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 907/1822] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:13:57 2022] PUT /identity/v3/projects/b2928d0b95a341a38549b76c06d6aa81/users/30c3ba6d924e4bb4a824d14b3d729984/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:57.394319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f33b8-6e1d-4f0f-83f3-798e38a1d92c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.394755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f33b8-6e1d-4f0f-83f3-798e38a1d92c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.394979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855f33b8-6e1d-4f0f-83f3-798e38a1d92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.427465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-855f33b8-6e1d-4f0f-83f3-798e38a1d92c None None] MFA Rules not processed for user `30c3ba6d924e4bb4a824d14b3d729984`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:57.428769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-855f33b8-6e1d-4f0f-83f3-798e38a1d92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:57.535343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 916/1823] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:57 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:13:57.544830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:57.547334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:57.618097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd7f621-d4c0-48b1-a77e-e6f5b6e0374b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:57.622486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0311a34b-31fa-47b2-8d44-cab54e6b94a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:57.623094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a34b-31fa-47b2-8d44-cab54e6b94a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.623462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a34b-31fa-47b2-8d44-cab54e6b94a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.623706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0311a34b-31fa-47b2-8d44-cab54e6b94a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.636764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311a34b-31fa-47b2-8d44-cab54e6b94a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:57.638900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0311a34b-31fa-47b2-8d44-cab54e6b94a4 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:57.657041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 908/1824] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:57.837526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f33b8-6e1d-4f0f-83f3-798e38a1d92c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:57.926103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:57.931569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855f33b8-6e1d-4f0f-83f3-798e38a1d92c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:57.943608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:57.944295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:57.944802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:57.945100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] PATH_INFO: `/v3/users/dac09bade8234939bcd4434e76cb97ae` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:57.946465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac09bade8234939bcd4434e76cb97ae)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:57.948718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:58.044001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac09bade8234939bcd4434e76cb97ae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.052873 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac09bade8234939bcd4434e76cb97ae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.053993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac09bade8234939bcd4434e76cb97ae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.065327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac09bade8234939bcd4434e76cb97ae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.066208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac09bade8234939bcd4434e76cb97ae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.093260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 917/1825] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:57 2022] DELETE /identity/v3/users/dac09bade8234939bcd4434e76cb97ae => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:58.097571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92d62-f428-428b-92b6-683e4c1d10be None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:58.097998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92d62-f428-428b-92b6-683e4c1d10be None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:58.098409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f92d62-f428-428b-92b6-683e4c1d10be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:58.118210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b2f92d62-f428-428b-92b6-683e4c1d10be None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:58.119672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b2f92d62-f428-428b-92b6-683e4c1d10be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:58.194184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 909/1826] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:58.204623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:58.206039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:58.279976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85d27a-5f1e-419f-a2a5-d8cc1cb13209 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:58.287933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6b6e5-7916-4f65-8e9c-b21162280c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:58.289502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6b6e5-7916-4f65-8e9c-b21162280c84 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:58.290193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6b6e5-7916-4f65-8e9c-b21162280c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:58.290998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6b6e5-7916-4f65-8e9c-b21162280c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:58.304331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6b6e5-7916-4f65-8e9c-b21162280c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:58.306714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6b6e5-7916-4f65-8e9c-b21162280c84 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:58.320749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 918/1827] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:13:58.686188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92d62-f428-428b-92b6-683e4c1d10be None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:58.687639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b2f92d62-f428-428b-92b6-683e4c1d10be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:58.720636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f92d62-f428-428b-92b6-683e4c1d10be None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:58.728064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:58.729061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:58.729394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:58.729675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] PATH_INFO: `/v3/projects/968e62e3cd394794bc673ec6d336d5e8` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:58.731123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] RBAC: Authorizing `identity:delete_project(project_id=968e62e3cd394794bc673ec6d336d5e8)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:58.733159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:58.822696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:58.845471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '968e62e3cd394794bc673ec6d336d5e8', 'request_id': 'req-82343c24-ebf6-488f-a1b8-62d7d7540216'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.858116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '968e62e3cd394794bc673ec6d336d5e8', 'request_id': 'req-82343c24-ebf6-488f-a1b8-62d7d7540216'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.880676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '968e62e3cd394794bc673ec6d336d5e8', 'request_id': 'req-82343c24-ebf6-488f-a1b8-62d7d7540216'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.897810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] The token cache is being invalidate because project 968e62e3cd394794bc673ec6d336d5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:58.898523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.899907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:58.902081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 910/1828] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:58 2022] DELETE /identity/v3/projects/968e62e3cd394794bc673ec6d336d5e8 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:58.905805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6b6e5-7916-4f65-8e9c-b21162280c84 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:58.920810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6b6e5-7916-4f65-8e9c-b21162280c84 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:59.059148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:59.060379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:59.060809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:59.061178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] PATH_INFO: `/v3/users/30c3ba6d924e4bb4a824d14b3d729984` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:59.063738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30c3ba6d924e4bb4a824d14b3d729984)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:59.065979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:59.102637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:13:59.146801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30c3ba6d924e4bb4a824d14b3d729984'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.159645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30c3ba6d924e4bb4a824d14b3d729984'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.160477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30c3ba6d924e4bb4a824d14b3d729984'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.180216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30c3ba6d924e4bb4a824d14b3d729984'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.181153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-25c6bfec-3920-43ba-a103-cfa721405ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30c3ba6d924e4bb4a824d14b3d729984'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.224019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 919/1829] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:13:58 2022] DELETE /identity/v3/users/30c3ba6d924e4bb4a824d14b3d729984 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:13:59.646649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:59.652260 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:59.677995 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82343c24-ebf6-488f-a1b8-62d7d7540216 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:13:59.784584 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811454e-1f0d-46cd-9d15-f2aa4d59a597 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:59.784948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811454e-1f0d-46cd-9d15-f2aa4d59a597 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:59.785373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5811454e-1f0d-46cd-9d15-f2aa4d59a597 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:59.787129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 920/1830] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:13:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 20 09:13:59.789099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:13:59.789991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:59.790269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:59.790542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] PATH_INFO: `/v3/projects/b2928d0b95a341a38549b76c06d6aa81` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:59.791792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2928d0b95a341a38549b76c06d6aa81)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:13:59.793904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:13:59.810493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609f372-a391-4510-a5bd-e82d81cb3e47 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:59.810875 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609f372-a391-4510-a5bd-e82d81cb3e47 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:59.811222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7609f372-a391-4510-a5bd-e82d81cb3e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:59.849034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-7609f372-a391-4510-a5bd-e82d81cb3e47 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:59.853882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7609f372-a391-4510-a5bd-e82d81cb3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:59.881793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2928d0b95a341a38549b76c06d6aa81', 'request_id': 'req-3cab8642-bad4-443f-afec-e9f407faa886'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.905376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2928d0b95a341a38549b76c06d6aa81', 'request_id': 'req-3cab8642-bad4-443f-afec-e9f407faa886'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.914055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2928d0b95a341a38549b76c06d6aa81', 'request_id': 'req-3cab8642-bad4-443f-afec-e9f407faa886'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.934101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] The token cache is being invalidate because project b2928d0b95a341a38549b76c06d6aa81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:13:59.934311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.934974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3cab8642-bad4-443f-afec-e9f407faa886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:13:59.937042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 911/1831] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:13:59 2022] DELETE /identity/v3/projects/b2928d0b95a341a38549b76c06d6aa81 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:13:59.941610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c148420-0b3c-44ba-9575-7cb54a5b7aa5 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:13:59.941890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c148420-0b3c-44ba-9575-7cb54a5b7aa5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:13:59.942168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c148420-0b3c-44ba-9575-7cb54a5b7aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:13:59.970788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 921/1832] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:13:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:13:59.984534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6c148420-0b3c-44ba-9575-7cb54a5b7aa5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:13:59.985800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6c148420-0b3c-44ba-9575-7cb54a5b7aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:13:59.994033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609f372-a391-4510-a5bd-e82d81cb3e47 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:13:59.996020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7609f372-a391-4510-a5bd-e82d81cb3e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:00.024091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7609f372-a391-4510-a5bd-e82d81cb3e47 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.031673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc070c-00d8-4ffc-99d2-3c9c1080750b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.032594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc070c-00d8-4ffc-99d2-3c9c1080750b service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.033092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc070c-00d8-4ffc-99d2-3c9c1080750b service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.033368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc070c-00d8-4ffc-99d2-3c9c1080750b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.047803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc070c-00d8-4ffc-99d2-3c9c1080750b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.050383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc070c-00d8-4ffc-99d2-3c9c1080750b service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.075839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 922/1833] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:13:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:14:00.085491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:00.108964 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 912/1834] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:00.116245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c551-43d0-42a8-af78-849edcd12c76 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.116537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c551-43d0-42a8-af78-849edcd12c76 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.116819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338c551-43d0-42a8-af78-849edcd12c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.137301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e338c551-43d0-42a8-af78-849edcd12c76 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:00.138756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e338c551-43d0-42a8-af78-849edcd12c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:00.152719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 923/1835] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:00.158094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c148420-0b3c-44ba-9575-7cb54a5b7aa5 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.167074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c148420-0b3c-44ba-9575-7cb54a5b7aa5 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.172517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.173202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.173609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.173884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.174627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:00.174931 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.176719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.190202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 913/1836] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:14:00.194638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338c551-43d0-42a8-af78-849edcd12c76 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.205513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338c551-43d0-42a8-af78-849edcd12c76 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.210458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3de28-03bf-4644-abce-63676cb0114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.211402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3de28-03bf-4644-abce-63676cb0114b admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.211600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3de28-03bf-4644-abce-63676cb0114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.212025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3de28-03bf-4644-abce-63676cb0114b admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.212753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3de28-03bf-4644-abce-63676cb0114b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.214729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3de28-03bf-4644-abce-63676cb0114b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.247705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 924/1837] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:00 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:00.251517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.262660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defee994-dfed-46d9-a301-534f42951223 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.326910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a298f6-8c99-4d27-817c-6255438b13cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.327711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a298f6-8c99-4d27-817c-6255438b13cb admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.328055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a298f6-8c99-4d27-817c-6255438b13cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.328394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a298f6-8c99-4d27-817c-6255438b13cb admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.329150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a298f6-8c99-4d27-817c-6255438b13cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.331123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a298f6-8c99-4d27-817c-6255438b13cb admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.353158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 914/1838] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:00 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:00.357349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3de28-03bf-4644-abce-63676cb0114b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.369713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3de28-03bf-4644-abce-63676cb0114b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.375988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc3ab3-b950-49b8-8790-4ebfafc6c50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.376797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc3ab3-b950-49b8-8790-4ebfafc6c50b admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.377091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc3ab3-b950-49b8-8790-4ebfafc6c50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.377365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fc3ab3-b950-49b8-8790-4ebfafc6c50b admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.378113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc3ab3-b950-49b8-8790-4ebfafc6c50b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.381610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fc3ab3-b950-49b8-8790-4ebfafc6c50b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.401754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 925/1839] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:00.402149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a298f6-8c99-4d27-817c-6255438b13cb admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.413695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a298f6-8c99-4d27-817c-6255438b13cb admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.420092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e523e68-ef87-4474-90b6-ec2f63698c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.421267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e523e68-ef87-4474-90b6-ec2f63698c8e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.421655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e523e68-ef87-4474-90b6-ec2f63698c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.422032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e523e68-ef87-4474-90b6-ec2f63698c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.423143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e523e68-ef87-4474-90b6-ec2f63698c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.425134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e523e68-ef87-4474-90b6-ec2f63698c8e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.437285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 915/1840] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:00.441428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc3ab3-b950-49b8-8790-4ebfafc6c50b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.451804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fc3ab3-b950-49b8-8790-4ebfafc6c50b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.458293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9124d-ab77-4ffc-a68f-79c58ebd4516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.459312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9124d-ab77-4ffc-a68f-79c58ebd4516 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.459697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9124d-ab77-4ffc-a68f-79c58ebd4516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.460037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9124d-ab77-4ffc-a68f-79c58ebd4516 admin admin] PATH_INFO: `/v3/projects/b8ae863007cd4c05b617a051f10d4610/users/08dc87ef53874406abb5c139f5f94100/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.478475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9124d-ab77-4ffc-a68f-79c58ebd4516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ae863007cd4c05b617a051f10d4610, user_id=08dc87ef53874406abb5c139f5f94100, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.481276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9124d-ab77-4ffc-a68f-79c58ebd4516 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.505619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 926/1841] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:00 2022] PUT /identity/v3/projects/b8ae863007cd4c05b617a051f10d4610/users/08dc87ef53874406abb5c139f5f94100/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:00.516085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b4e9b-1164-46f5-9b71-7384fe87d5fb None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.516380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b4e9b-1164-46f5-9b71-7384fe87d5fb None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.516664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b4e9b-1164-46f5-9b71-7384fe87d5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.545200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-5d3b4e9b-1164-46f5-9b71-7384fe87d5fb None None] MFA Rules not processed for user `08dc87ef53874406abb5c139f5f94100`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:00.546869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5d3b4e9b-1164-46f5-9b71-7384fe87d5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:00.623971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 916/1842] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:00.628523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9124d-ab77-4ffc-a68f-79c58ebd4516 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.642287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:00.693964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9124d-ab77-4ffc-a68f-79c58ebd4516 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.698702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf33ca-0e18-44c5-8ea0-f8faa4d744d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.699374 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf33ca-0e18-44c5-8ea0-f8faa4d744d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.699689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf33ca-0e18-44c5-8ea0-f8faa4d744d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.699951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf33ca-0e18-44c5-8ea0-f8faa4d744d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.700635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf33ca-0e18-44c5-8ea0-f8faa4d744d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.702548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf33ca-0e18-44c5-8ea0-f8faa4d744d0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.731888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 927/1843] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:00 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:00.737744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b4e9b-1164-46f5-9b71-7384fe87d5fb None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.750783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b4e9b-1164-46f5-9b71-7384fe87d5fb None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.800091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:00.811667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2776e-c928-49ed-b96c-355417b8f0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.812375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2776e-c928-49ed-b96c-355417b8f0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.812662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2776e-c928-49ed-b96c-355417b8f0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.812944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2776e-c928-49ed-b96c-355417b8f0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.813656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2776e-c928-49ed-b96c-355417b8f0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.815656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b2776e-c928-49ed-b96c-355417b8f0a7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.837144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 917/1844] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:00 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:00.841836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf33ca-0e18-44c5-8ea0-f8faa4d744d0 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.852335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf33ca-0e18-44c5-8ea0-f8faa4d744d0 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.858035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f13bc5-4354-4f72-9d62-0712c3a488ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.858690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f13bc5-4354-4f72-9d62-0712c3a488ab admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.858992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f13bc5-4354-4f72-9d62-0712c3a488ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.859279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f13bc5-4354-4f72-9d62-0712c3a488ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.860025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f13bc5-4354-4f72-9d62-0712c3a488ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.861611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f13bc5-4354-4f72-9d62-0712c3a488ab admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.872504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 928/1845] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:00.876811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2776e-c928-49ed-b96c-355417b8f0a7 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.887054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2776e-c928-49ed-b96c-355417b8f0a7 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:00.892301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b0b36-f465-41ff-abc1-c7cc27768404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:00.893190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b0b36-f465-41ff-abc1-c7cc27768404 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:00.893469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b0b36-f465-41ff-abc1-c7cc27768404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:00.893746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b0b36-f465-41ff-abc1-c7cc27768404 admin admin] PATH_INFO: `/v3/projects/862b62a96df8476e9a2f26cededed3f7/users/ce04fa44a47a442da93db73d0240f37c/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:00.909434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b0b36-f465-41ff-abc1-c7cc27768404 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862b62a96df8476e9a2f26cededed3f7, user_id=ce04fa44a47a442da93db73d0240f37c, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:00.912154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3b0b36-f465-41ff-abc1-c7cc27768404 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:00.927345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 918/1846] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:00 2022] PUT /identity/v3/projects/862b62a96df8476e9a2f26cededed3f7/users/ce04fa44a47a442da93db73d0240f37c/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:00.931947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f13bc5-4354-4f72-9d62-0712c3a488ab admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:00.946454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f13bc5-4354-4f72-9d62-0712c3a488ab admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:01.009719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98b840-84e1-4ee2-808c-85f59811636a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:01.010377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98b840-84e1-4ee2-808c-85f59811636a admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:01.010689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98b840-84e1-4ee2-808c-85f59811636a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:01.010960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98b840-84e1-4ee2-808c-85f59811636a admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:01.011728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98b840-84e1-4ee2-808c-85f59811636a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:01.013432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e98b840-84e1-4ee2-808c-85f59811636a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:01.023120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 929/1847] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:01.028044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b0b36-f465-41ff-abc1-c7cc27768404 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:01.038573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b0b36-f465-41ff-abc1-c7cc27768404 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:01.043645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:01.044549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:01.044827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:01.045096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] PATH_INFO: `/v3/projects/862b62a96df8476e9a2f26cededed3f7/users/ce04fa44a47a442da93db73d0240f37c/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:01.047525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=862b62a96df8476e9a2f26cededed3f7, user_id=ce04fa44a47a442da93db73d0240f37c, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:01.050153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:01.061195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 919/1848] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:01 2022] PUT /identity/v3/projects/862b62a96df8476e9a2f26cededed3f7/users/ce04fa44a47a442da93db73d0240f37c/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:01.072349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4bb1f-1cf9-43c5-923c-73719ea3f0f6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:01.072643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4bb1f-1cf9-43c5-923c-73719ea3f0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:01.072916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4bb1f-1cf9-43c5-923c-73719ea3f0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:01.102472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-20d4bb1f-1cf9-43c5-923c-73719ea3f0f6 None None] MFA Rules not processed for user `ce04fa44a47a442da93db73d0240f37c`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:01.103719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-20d4bb1f-1cf9-43c5-923c-73719ea3f0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:01.176756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 930/1849] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:01 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:01.189209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:01.191053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:01.250181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9650b75-c1f2-4702-94b9-c5efb5246dd5 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:01.254090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc37c57-f2b5-4b8b-9e31-e8572e678646 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:01.254636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc37c57-f2b5-4b8b-9e31-e8572e678646 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:01.254914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc37c57-f2b5-4b8b-9e31-e8572e678646 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:01.255189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc37c57-f2b5-4b8b-9e31-e8572e678646 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:01.265430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc37c57-f2b5-4b8b-9e31-e8572e678646 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:01.266880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc37c57-f2b5-4b8b-9e31-e8572e678646 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:01.276647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 920/1850] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:01 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:01.942258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4bb1f-1cf9-43c5-923c-73719ea3f0f6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:01.958379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:02.063988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4bb1f-1cf9-43c5-923c-73719ea3f0f6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:02.069959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:02.070917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:02.071216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:02.071532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] PATH_INFO: `/v3/users/08dc87ef53874406abb5c139f5f94100` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:02.072808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08dc87ef53874406abb5c139f5f94100)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:02.074939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:02.138658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08dc87ef53874406abb5c139f5f94100'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:02.148455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08dc87ef53874406abb5c139f5f94100'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:02.149353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08dc87ef53874406abb5c139f5f94100'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:02.164122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08dc87ef53874406abb5c139f5f94100'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:02.165019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08dc87ef53874406abb5c139f5f94100'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:02.188346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 931/1851] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:01 2022] DELETE /identity/v3/users/08dc87ef53874406abb5c139f5f94100 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:02.192863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f83656-942a-4db7-8649-da581c8770f9 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:02.193352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f83656-942a-4db7-8649-da581c8770f9 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:02.193352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f83656-942a-4db7-8649-da581c8770f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:02.212463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-95f83656-942a-4db7-8649-da581c8770f9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:02.213733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-95f83656-942a-4db7-8649-da581c8770f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:02.289830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 921/1852] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:02.299367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:02.300858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:02.374370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28511b2-3525-4473-a139-87158e50ecc2 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:02.379129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09626d-681c-4da3-8230-6b3b49ccec05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:02.380066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09626d-681c-4da3-8230-6b3b49ccec05 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:02.380500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09626d-681c-4da3-8230-6b3b49ccec05 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:02.380903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb09626d-681c-4da3-8230-6b3b49ccec05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:02.397705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09626d-681c-4da3-8230-6b3b49ccec05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:02.400326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb09626d-681c-4da3-8230-6b3b49ccec05 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:02.417287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 932/1853] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:03.112154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f83656-942a-4db7-8649-da581c8770f9 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:03.113796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-95f83656-942a-4db7-8649-da581c8770f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:03.126991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f83656-942a-4db7-8649-da581c8770f9 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:03.132098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:03.133431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:03.134296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:03.134296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] PATH_INFO: `/v3/projects/b8ae863007cd4c05b617a051f10d4610` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:03.135566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ae863007cd4c05b617a051f10d4610)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:03.137673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:03.198047 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:03.201916 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ae863007cd4c05b617a051f10d4610', 'request_id': 'req-f7784448-f7bf-4271-a6ba-3477256f7cac'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.215930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ae863007cd4c05b617a051f10d4610', 'request_id': 'req-f7784448-f7bf-4271-a6ba-3477256f7cac'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.223839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ae863007cd4c05b617a051f10d4610', 'request_id': 'req-f7784448-f7bf-4271-a6ba-3477256f7cac'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.236083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] The token cache is being invalidate because project b8ae863007cd4c05b617a051f10d4610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:03.236375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.237032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.238924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 922/1854] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:03 2022] DELETE /identity/v3/projects/b8ae863007cd4c05b617a051f10d4610 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:03.242626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09626d-681c-4da3-8230-6b3b49ccec05 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:03.252587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb09626d-681c-4da3-8230-6b3b49ccec05 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:03.280887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:03.309478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:03.310355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:03.310639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:03.310917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] PATH_INFO: `/v3/users/ce04fa44a47a442da93db73d0240f37c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:03.312143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce04fa44a47a442da93db73d0240f37c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:03.314079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:03.371421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce04fa44a47a442da93db73d0240f37c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.379565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce04fa44a47a442da93db73d0240f37c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.380269 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce04fa44a47a442da93db73d0240f37c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.390260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce04fa44a47a442da93db73d0240f37c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.390992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cbd29344-b50a-4037-a96f-8324fb3958a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce04fa44a47a442da93db73d0240f37c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.409318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 933/1855] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:03 2022] DELETE /identity/v3/users/ce04fa44a47a442da93db73d0240f37c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:03.688638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:03.690300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:03.713342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7784448-f7bf-4271-a6ba-3477256f7cac admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:03.770348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:03.771237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:03.771548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:03.771859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] PATH_INFO: `/v3/projects/862b62a96df8476e9a2f26cededed3f7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:03.773275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] RBAC: Authorizing `identity:delete_project(project_id=862b62a96df8476e9a2f26cededed3f7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:03.775373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:03.823771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '862b62a96df8476e9a2f26cededed3f7', 'request_id': 'req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.833282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '862b62a96df8476e9a2f26cededed3f7', 'request_id': 'req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.838831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '862b62a96df8476e9a2f26cededed3f7', 'request_id': 'req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.849667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] The token cache is being invalidate because project 862b62a96df8476e9a2f26cededed3f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:03.849956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.850595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cd5bc5d9-9af6-42e4-bb55-0c3a5166130e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:03.852176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 923/1856] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:03 2022] DELETE /identity/v3/projects/862b62a96df8476e9a2f26cededed3f7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:03.857380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e50134-ac1c-418a-a425-309a2545f288 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:03.857668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e50134-ac1c-418a-a425-309a2545f288 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:03.857975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e50134-ac1c-418a-a425-309a2545f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:03.876064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f5e50134-ac1c-418a-a425-309a2545f288 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:03.877262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f5e50134-ac1c-418a-a425-309a2545f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:03.949411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 934/1857] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:03.958189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9591198-0ab4-426c-8df3-816a222df12e None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:03.958486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9591198-0ab4-426c-8df3-816a222df12e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:03.958846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9591198-0ab4-426c-8df3-816a222df12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:03.983775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f9591198-0ab4-426c-8df3-816a222df12e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:03.984976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f9591198-0ab4-426c-8df3-816a222df12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:03.993617 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 924/1858] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:03.998480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e50134-ac1c-418a-a425-309a2545f288 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:04.008981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e50134-ac1c-418a-a425-309a2545f288 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:04.014897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:04.015582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:04.015863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:04.016139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:04.016868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:04.017175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:04.018793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:04.031006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 935/1859] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:14:04.035431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9591198-0ab4-426c-8df3-816a222df12e None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:04.045813 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9591198-0ab4-426c-8df3-816a222df12e None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:04.070113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840a35c-ffb8-44b0-83de-baf26fe9fb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:04.071045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840a35c-ffb8-44b0-83de-baf26fe9fb4a admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:04.071365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840a35c-ffb8-44b0-83de-baf26fe9fb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:04.071651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c840a35c-ffb8-44b0-83de-baf26fe9fb4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:04.072369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840a35c-ffb8-44b0-83de-baf26fe9fb4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:04.074401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c840a35c-ffb8-44b0-83de-baf26fe9fb4a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:04.121463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 925/1860] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:04 2022] POST /identity/v3/projects => generated 385 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:04.122752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:04.134175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe39e6-2156-4a3b-85f4-0cd29a6bfd8b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:04.170272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:04.198632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e350df-005c-415d-af8d-fb707470482c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:04.199695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e350df-005c-415d-af8d-fb707470482c admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:04.200426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e350df-005c-415d-af8d-fb707470482c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:04.200845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e350df-005c-415d-af8d-fb707470482c admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:04.201773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e350df-005c-415d-af8d-fb707470482c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:04.204165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e350df-005c-415d-af8d-fb707470482c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:04.238461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 936/1861] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:04 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:04.240599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840a35c-ffb8-44b0-83de-baf26fe9fb4a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:04.253201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c840a35c-ffb8-44b0-83de-baf26fe9fb4a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:04.259721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a28a-2669-43d9-b6d3-d5b931828fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:04.260507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a28a-2669-43d9-b6d3-d5b931828fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:04.261140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a28a-2669-43d9-b6d3-d5b931828fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:04.261442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0a28a-2669-43d9-b6d3-d5b931828fa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:04.262268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0a28a-2669-43d9-b6d3-d5b931828fa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:04.264559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0a28a-2669-43d9-b6d3-d5b931828fa7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:04.277231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 926/1862] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:04.281547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e350df-005c-415d-af8d-fb707470482c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:04.293610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e350df-005c-415d-af8d-fb707470482c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:04.300237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ce32c-429f-4568-912e-ac563f6ef66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:04.301016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce32c-429f-4568-912e-ac563f6ef66e admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:04.301302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce32c-429f-4568-912e-ac563f6ef66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:04.301581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ce32c-429f-4568-912e-ac563f6ef66e admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:04.302443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ce32c-429f-4568-912e-ac563f6ef66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:04.304396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ce32c-429f-4568-912e-ac563f6ef66e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:04.316444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 937/1863] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:04.320395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a28a-2669-43d9-b6d3-d5b931828fa7 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:04.331171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0a28a-2669-43d9-b6d3-d5b931828fa7 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:04.336421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:04.337492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:04.337927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:04.338346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] PATH_INFO: `/v3/projects/60d91d76cb44401a8da94f03d534d52b/users/8c344cf796654995b3c610a7dd4ca059/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:04.355849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60d91d76cb44401a8da94f03d534d52b, user_id=8c344cf796654995b3c610a7dd4ca059, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:04.358831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:04.369506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:04.377142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 927/1864] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:04 2022] PUT /identity/v3/projects/60d91d76cb44401a8da94f03d534d52b/users/8c344cf796654995b3c610a7dd4ca059/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:04.387986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e490bea-c52d-4268-a415-ffd7b8fbab40 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:04.388776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e490bea-c52d-4268-a415-ffd7b8fbab40 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:04.389216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e490bea-c52d-4268-a415-ffd7b8fbab40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:04.423835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5e490bea-c52d-4268-a415-ffd7b8fbab40 None None] MFA Rules not processed for user `8c344cf796654995b3c610a7dd4ca059`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:04.425234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5e490bea-c52d-4268-a415-ffd7b8fbab40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:04.556922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 938/1865] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:04.559996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:04.564355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:04.717034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84d90c9-49bd-402f-a3bf-fd75cc2773d4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:04.728200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4d3ccd-f76e-4f3d-bf92-43f27716b8a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:04.730119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d3ccd-f76e-4f3d-bf92-43f27716b8a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:04.730862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d3ccd-f76e-4f3d-bf92-43f27716b8a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:04.732004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4d3ccd-f76e-4f3d-bf92-43f27716b8a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:04.765040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d3ccd-f76e-4f3d-bf92-43f27716b8a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:04.781827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4d3ccd-f76e-4f3d-bf92-43f27716b8a6 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:04.796379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 928/1866] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:04 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:14.538999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=49,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:14.789003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=27 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:14.969505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e490bea-c52d-4268-a415-ffd7b8fbab40 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:15.035908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e490bea-c52d-4268-a415-ffd7b8fbab40 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:15.040627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:15.041505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:15.041785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:15.042059 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] PATH_INFO: `/v3/users/8c344cf796654995b3c610a7dd4ca059` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:15.043283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c344cf796654995b3c610a7dd4ca059)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:15.045378 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:15.097380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c344cf796654995b3c610a7dd4ca059'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.104194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c344cf796654995b3c610a7dd4ca059'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.104861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c344cf796654995b3c610a7dd4ca059'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.113230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c344cf796654995b3c610a7dd4ca059'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.114102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c344cf796654995b3c610a7dd4ca059'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.132262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 939/1867] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:14 2022] DELETE /identity/v3/users/8c344cf796654995b3c610a7dd4ca059 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:15.137123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a2bde-cdb6-489d-abe7-6e437f3ac7aa None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:15.137413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a2bde-cdb6-489d-abe7-6e437f3ac7aa None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:15.137696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a2bde-cdb6-489d-abe7-6e437f3ac7aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:15.163017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e95a2bde-cdb6-489d-abe7-6e437f3ac7aa None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:15.164296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e95a2bde-cdb6-489d-abe7-6e437f3ac7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:15.228020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 929/1868] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:15.238578 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:15.241186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:15.301973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e0dc-c5ba-4265-a3e8-65371ebb8a5b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:15.305819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba0d87-3f65-4628-b8cb-deda8a8b06f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:15.306314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0d87-3f65-4628-b8cb-deda8a8b06f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:15.306637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0d87-3f65-4628-b8cb-deda8a8b06f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:15.306907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba0d87-3f65-4628-b8cb-deda8a8b06f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:15.316062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba0d87-3f65-4628-b8cb-deda8a8b06f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:15.317492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba0d87-3f65-4628-b8cb-deda8a8b06f6 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:15.328499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 940/1869] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:14:15.759280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a2bde-cdb6-489d-abe7-6e437f3ac7aa None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:15.760758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e95a2bde-cdb6-489d-abe7-6e437f3ac7aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:15.783073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a2bde-cdb6-489d-abe7-6e437f3ac7aa None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:15.787901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:15.788778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:15.789054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:15.789909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] PATH_INFO: `/v3/projects/60d91d76cb44401a8da94f03d534d52b` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:15.791140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] RBAC: Authorizing `identity:delete_project(project_id=60d91d76cb44401a8da94f03d534d52b)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:15.793075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:15.852338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60d91d76cb44401a8da94f03d534d52b', 'request_id': 'req-6fd99707-3d26-4263-ac4b-398fe29d371b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.860922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60d91d76cb44401a8da94f03d534d52b', 'request_id': 'req-6fd99707-3d26-4263-ac4b-398fe29d371b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.866608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60d91d76cb44401a8da94f03d534d52b', 'request_id': 'req-6fd99707-3d26-4263-ac4b-398fe29d371b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.878025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] The token cache is being invalidate because project 60d91d76cb44401a8da94f03d534d52b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:15.878295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.878972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6fd99707-3d26-4263-ac4b-398fe29d371b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:15.880668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 930/1870] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:15 2022] DELETE /identity/v3/projects/60d91d76cb44401a8da94f03d534d52b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:15.885661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60514d-c9ce-4eaa-83b6-f488161c69fc None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:15.886145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60514d-c9ce-4eaa-83b6-f488161c69fc None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:15.886231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc60514d-c9ce-4eaa-83b6-f488161c69fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:15.905304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-dc60514d-c9ce-4eaa-83b6-f488161c69fc None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:15.906494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-dc60514d-c9ce-4eaa-83b6-f488161c69fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:15.923640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:15.978149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 941/1871] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:15.985968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9edfa-eb19-49ae-a311-d7b08e77cca4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:15.986251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9edfa-eb19-49ae-a311-d7b08e77cca4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:15.986527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9edfa-eb19-49ae-a311-d7b08e77cca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.005832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-32b9edfa-eb19-49ae-a311-d7b08e77cca4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:16.007055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-32b9edfa-eb19-49ae-a311-d7b08e77cca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:16.015175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 931/1872] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:16.019538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60514d-c9ce-4eaa-83b6-f488161c69fc None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:16.029832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc60514d-c9ce-4eaa-83b6-f488161c69fc None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:16.034836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:16.035569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:16.035853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:16.036113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.037034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:16.037506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:16.039315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:16.052434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 942/1873] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:14:16.057029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9edfa-eb19-49ae-a311-d7b08e77cca4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:16.066448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9edfa-eb19-49ae-a311-d7b08e77cca4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:16.073060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc820062-42af-4cfa-b0f7-da79024da742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:16.073506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820062-42af-4cfa-b0f7-da79024da742 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:16.073791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820062-42af-4cfa-b0f7-da79024da742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:16.074168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc820062-42af-4cfa-b0f7-da79024da742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.074782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc820062-42af-4cfa-b0f7-da79024da742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:16.076812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc820062-42af-4cfa-b0f7-da79024da742 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:16.106632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 932/1874] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:16 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:16.107915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:16.119621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b7022e-d7df-4069-9292-dbbd841700a7 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:16.179865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862394b9-7295-48bf-a65e-43b46a137e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:16.180551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862394b9-7295-48bf-a65e-43b46a137e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:16.180864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862394b9-7295-48bf-a65e-43b46a137e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:16.181169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862394b9-7295-48bf-a65e-43b46a137e8b admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.181865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862394b9-7295-48bf-a65e-43b46a137e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:16.183834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862394b9-7295-48bf-a65e-43b46a137e8b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:16.207459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 943/1875] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:16 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:16.208676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc820062-42af-4cfa-b0f7-da79024da742 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:16.219666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc820062-42af-4cfa-b0f7-da79024da742 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:16.224360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479c00-f5f1-45c3-bab8-ba4a880578e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:16.225070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479c00-f5f1-45c3-bab8-ba4a880578e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:16.225354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479c00-f5f1-45c3-bab8-ba4a880578e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:16.225630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c479c00-f5f1-45c3-bab8-ba4a880578e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.226368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c479c00-f5f1-45c3-bab8-ba4a880578e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:16.228037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c479c00-f5f1-45c3-bab8-ba4a880578e4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:16.238736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 933/1876] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:16.242824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862394b9-7295-48bf-a65e-43b46a137e8b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:16.253070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862394b9-7295-48bf-a65e-43b46a137e8b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:16.257999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a3d417-485c-4163-9c42-26d5f0b549d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:16.258667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3d417-485c-4163-9c42-26d5f0b549d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:16.258947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3d417-485c-4163-9c42-26d5f0b549d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:16.259224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a3d417-485c-4163-9c42-26d5f0b549d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.260038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3d417-485c-4163-9c42-26d5f0b549d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:16.261631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a3d417-485c-4163-9c42-26d5f0b549d9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:16.272155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 944/1877] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:16.277103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479c00-f5f1-45c3-bab8-ba4a880578e4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:16.287563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c479c00-f5f1-45c3-bab8-ba4a880578e4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:16.292471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:16.293386 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:16.293665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:16.293938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] PATH_INFO: `/v3/projects/7264062888b544cfb7fbc74cc87f386f/users/712133b246c94438b94c1f865dca1be9/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.310277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7264062888b544cfb7fbc74cc87f386f, user_id=712133b246c94438b94c1f865dca1be9, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:16.313105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:16.328256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 934/1878] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:16 2022] PUT /identity/v3/projects/7264062888b544cfb7fbc74cc87f386f/users/712133b246c94438b94c1f865dca1be9/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:16.333469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cea42-0928-4ee2-94fa-bbfdb392a407 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:16.333761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cea42-0928-4ee2-94fa-bbfdb392a407 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:16.334049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38cea42-0928-4ee2-94fa-bbfdb392a407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.352477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b38cea42-0928-4ee2-94fa-bbfdb392a407 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:16.353686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b38cea42-0928-4ee2-94fa-bbfdb392a407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:16.414817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 945/1879] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:16.424692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:16.426669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:16.449397 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:16.486541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e5cb-935d-4e3b-93b5-989c701ffda4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:16.490537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:16.491121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:16.491449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:16.491710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:16.501059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:16.502479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:16.512044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 935/1880] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:17.186941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef502-246b-41eb-a8d2-dc09dcfbb13a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:17.187535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef502-246b-41eb-a8d2-dc09dcfbb13a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:17.187535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66ef502-246b-41eb-a8d2-dc09dcfbb13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:17.216547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d66ef502-246b-41eb-a8d2-dc09dcfbb13a None None] MFA Rules not processed for user `712133b246c94438b94c1f865dca1be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:17.217792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d66ef502-246b-41eb-a8d2-dc09dcfbb13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:17.235399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:17.288956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 946/1881] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:17.300292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:17.301893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:17.370000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c91768a-2151-4012-8031-ebf2b007c0e0 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:17.374046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:17.374732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:17.374992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:17.375364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:17.385631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:17.388278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:17.400816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 936/1882] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:18.766652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a915d-6ac9-473f-9697-e67ad2a03ab0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:18.767077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a915d-6ac9-473f-9697-e67ad2a03ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:18.767228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a915d-6ac9-473f-9697-e67ad2a03ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:18.793195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a63a915d-6ac9-473f-9697-e67ad2a03ab0 None None] MFA Rules not processed for user `712133b246c94438b94c1f865dca1be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:18.796525 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a63a915d-6ac9-473f-9697-e67ad2a03ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:18.806975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 947/1883] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:23.331730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:23.333253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:23.348120 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6d893-ddf6-44d9-835b-a6c5b95e765a service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:23.353005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:23.353875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:23.354157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:23.354436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] PATH_INFO: `/v3/users/3289ac584c134566b6378c1de33f669c` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:23.355763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3289ac584c134566b6378c1de33f669c)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:23.357825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:23.407817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3289ac584c134566b6378c1de33f669c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:23.408503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3289ac584c134566b6378c1de33f669c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:23.409094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3289ac584c134566b6378c1de33f669c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:23.417598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3289ac584c134566b6378c1de33f669c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:23.425221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3289ac584c134566b6378c1de33f669c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:23.442675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 937/1884] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:23 2022] DELETE /identity/v3/users/3289ac584c134566b6378c1de33f669c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:23.446990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe28c8-95d0-4f08-a122-fa2226b8cca1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:23.447484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe28c8-95d0-4f08-a122-fa2226b8cca1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:23.447592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fe28c8-95d0-4f08-a122-fa2226b8cca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:23.473296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-63fe28c8-95d0-4f08-a122-fa2226b8cca1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:23.474862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-63fe28c8-95d0-4f08-a122-fa2226b8cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:23.619408 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 948/1885] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:23.640703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:23.642208 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:23.706380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:23.725596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb144328-5171-4ed7-93f3-9dc56c452e26 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:23.730256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d84add-9eda-4e9c-af9c-cb788111b0c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:23.730939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d84add-9eda-4e9c-af9c-cb788111b0c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:23.731220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d84add-9eda-4e9c-af9c-cb788111b0c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:23.731526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d84add-9eda-4e9c-af9c-cb788111b0c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:23.743141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d84add-9eda-4e9c-af9c-cb788111b0c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:23.745440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d84add-9eda-4e9c-af9c-cb788111b0c8 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:23.758447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 938/1886] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:24.384647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe28c8-95d0-4f08-a122-fa2226b8cca1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:24.386080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-63fe28c8-95d0-4f08-a122-fa2226b8cca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:24.399960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fe28c8-95d0-4f08-a122-fa2226b8cca1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:24.405359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:24.406227 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:24.406530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:24.406797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] PATH_INFO: `/v3/projects/d0bcb0fa0fd644feaad5ffe766b47c04` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:24.408013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0bcb0fa0fd644feaad5ffe766b47c04)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:24.409924 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:24.459207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0bcb0fa0fd644feaad5ffe766b47c04', 'request_id': 'req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.464873 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0bcb0fa0fd644feaad5ffe766b47c04', 'request_id': 'req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.471188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0bcb0fa0fd644feaad5ffe766b47c04', 'request_id': 'req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.484760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] The token cache is being invalidate because project d0bcb0fa0fd644feaad5ffe766b47c04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:24.485051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.485679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.487397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 949/1887] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:24 2022] DELETE /identity/v3/projects/d0bcb0fa0fd644feaad5ffe766b47c04 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:24.491370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d84add-9eda-4e9c-af9c-cb788111b0c8 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:24.501506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d84add-9eda-4e9c-af9c-cb788111b0c8 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:24.561003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:24.561897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:24.562180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:24.562450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] PATH_INFO: `/v3/users/258361576e804fb596e22af3d12e7856` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:24.563720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] RBAC: Authorizing `identity:delete_user(user_id=258361576e804fb596e22af3d12e7856)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:24.565650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:24.619356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '258361576e804fb596e22af3d12e7856'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.620017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '258361576e804fb596e22af3d12e7856'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.620594 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '258361576e804fb596e22af3d12e7856'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.630156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '258361576e804fb596e22af3d12e7856'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.638033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a9c768d-2cae-45f4-9844-8fa713f4b417 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '258361576e804fb596e22af3d12e7856'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:24.656535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 939/1888] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:24 2022] DELETE /identity/v3/users/258361576e804fb596e22af3d12e7856 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:25.074889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:25.076606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:25.100494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8c9d17-b218-41b8-a6be-75e003ad4f3d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:25.154260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:25.191007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:25.192138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.192529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.193041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] PATH_INFO: `/v3/projects/e9579225e79b44babbfdba246b229acb` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.194471 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9579225e79b44babbfdba246b229acb)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:25.196938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:25.268883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9579225e79b44babbfdba246b229acb', 'request_id': 'req-5587e230-f76a-45ac-aced-b433a24b1ec4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:25.277387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9579225e79b44babbfdba246b229acb', 'request_id': 'req-5587e230-f76a-45ac-aced-b433a24b1ec4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:25.284565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9579225e79b44babbfdba246b229acb', 'request_id': 'req-5587e230-f76a-45ac-aced-b433a24b1ec4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:25.308755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] The token cache is being invalidate because project e9579225e79b44babbfdba246b229acb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:25.309117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:25.310612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-5587e230-f76a-45ac-aced-b433a24b1ec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:25.315468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 950/1889] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:25 2022] DELETE /identity/v3/projects/e9579225e79b44babbfdba246b229acb => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:25.318754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8d72-6e6a-481b-b0b3-ee3f32465c9a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.319097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8d72-6e6a-481b-b0b3-ee3f32465c9a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.319517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fd8d72-6e6a-481b-b0b3-ee3f32465c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.346012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b2fd8d72-6e6a-481b-b0b3-ee3f32465c9a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:25.347239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b2fd8d72-6e6a-481b-b0b3-ee3f32465c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:25.455196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:25.474773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 940/1890] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:25.475878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79b1f6-82e6-435c-b164-72915b784431 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.477105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79b1f6-82e6-435c-b164-72915b784431 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.477313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79b1f6-82e6-435c-b164-72915b784431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.511549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-9e79b1f6-82e6-435c-b164-72915b784431 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:25.513571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9e79b1f6-82e6-435c-b164-72915b784431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:25.531848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 951/1891] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:25.534032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd8d72-6e6a-481b-b0b3-ee3f32465c9a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:25.559740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fd8d72-6e6a-481b-b0b3-ee3f32465c9a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:25.568955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:25.569895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.570275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.570955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.572000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:25.572667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:25.574472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:25.594881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 941/1892] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:14:25.595304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79b1f6-82e6-435c-b164-72915b784431 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:25.609550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79b1f6-82e6-435c-b164-72915b784431 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:25.616773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb22d3-b8d2-4bee-a5ad-7f4c67a24969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:25.618635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb22d3-b8d2-4bee-a5ad-7f4c67a24969 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.618864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb22d3-b8d2-4bee-a5ad-7f4c67a24969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.619089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb22d3-b8d2-4bee-a5ad-7f4c67a24969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.620232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb22d3-b8d2-4bee-a5ad-7f4c67a24969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:25.625370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb22d3-b8d2-4bee-a5ad-7f4c67a24969 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:25.657216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 952/1893] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:25 2022] POST /identity/v3/projects => generated 367 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:25.661165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:25.675383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b711f9f-2ef3-4f86-9e20-348d903a17cd admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:25.823870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb64e8-8732-4b66-8122-73cf470e6fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:25.824560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb64e8-8732-4b66-8122-73cf470e6fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.824851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb64e8-8732-4b66-8122-73cf470e6fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.825119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bb64e8-8732-4b66-8122-73cf470e6fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.825837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb64e8-8732-4b66-8122-73cf470e6fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:25.828973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bb64e8-8732-4b66-8122-73cf470e6fd0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:25.853039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 942/1894] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:25 2022] POST /identity/v3/users => generated 284 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:25.857384 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb22d3-b8d2-4bee-a5ad-7f4c67a24969 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:25.868429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb22d3-b8d2-4bee-a5ad-7f4c67a24969 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:25.873222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b66f39-b14e-4ca3-b767-f88401c6e2db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:25.874011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66f39-b14e-4ca3-b767-f88401c6e2db admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.874463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66f39-b14e-4ca3-b767-f88401c6e2db admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.874763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b66f39-b14e-4ca3-b767-f88401c6e2db admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.875576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b66f39-b14e-4ca3-b767-f88401c6e2db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:25.877216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b66f39-b14e-4ca3-b767-f88401c6e2db admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:25.887642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 953/1895] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:25.891738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb64e8-8732-4b66-8122-73cf470e6fd0 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:25.902283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bb64e8-8732-4b66-8122-73cf470e6fd0 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:25.907126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0a795-5fb4-4314-bddb-464519e3385e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:25.907837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0a795-5fb4-4314-bddb-464519e3385e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.908123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0a795-5fb4-4314-bddb-464519e3385e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.908526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0a795-5fb4-4314-bddb-464519e3385e admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.909267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0a795-5fb4-4314-bddb-464519e3385e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:25.910905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0a795-5fb4-4314-bddb-464519e3385e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:25.921508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 943/1896] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:25.925818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b66f39-b14e-4ca3-b767-f88401c6e2db admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:25.935931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b66f39-b14e-4ca3-b767-f88401c6e2db admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:25.940819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cb058-d8af-4c58-acd3-13583a717cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:25.941701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cb058-d8af-4c58-acd3-13583a717cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.941983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cb058-d8af-4c58-acd3-13583a717cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.942262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cb058-d8af-4c58-acd3-13583a717cdd admin admin] PATH_INFO: `/v3/projects/435fb0a7764c467889c0c107f9d3455d/users/0419dec18e67414882042dd3e872f5fa/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:25.956973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cb058-d8af-4c58-acd3-13583a717cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=435fb0a7764c467889c0c107f9d3455d, user_id=0419dec18e67414882042dd3e872f5fa, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:25.959671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cb058-d8af-4c58-acd3-13583a717cdd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:25.974757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 954/1897] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:25 2022] PUT /identity/v3/projects/435fb0a7764c467889c0c107f9d3455d/users/0419dec18e67414882042dd3e872f5fa/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:25.986086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e129f-1e97-41af-9470-27cb8045e5dd None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:25.986486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e129f-1e97-41af-9470-27cb8045e5dd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:25.986833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848e129f-1e97-41af-9470-27cb8045e5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.013957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-848e129f-1e97-41af-9470-27cb8045e5dd None None] MFA Rules not processed for user `0419dec18e67414882042dd3e872f5fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:26.015592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-848e129f-1e97-41af-9470-27cb8045e5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:26.084741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 944/1898] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:26.089536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cb058-d8af-4c58-acd3-13583a717cdd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:26.153478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cb058-d8af-4c58-acd3-13583a717cdd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:26.158277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67734851-41d1-4624-afa8-a10ae993ec6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:26.158936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67734851-41d1-4624-afa8-a10ae993ec6c admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.159219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67734851-41d1-4624-afa8-a10ae993ec6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.159532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67734851-41d1-4624-afa8-a10ae993ec6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.160217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67734851-41d1-4624-afa8-a10ae993ec6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:26.162123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67734851-41d1-4624-afa8-a10ae993ec6c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:26.166184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:26.191395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 955/1899] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:26 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:26.195391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e129f-1e97-41af-9470-27cb8045e5dd None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:26.206758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848e129f-1e97-41af-9470-27cb8045e5dd None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:26.258233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:26.267168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b086a92-e877-4d67-b76d-9eccb45957e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:26.267976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b086a92-e877-4d67-b76d-9eccb45957e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.268203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b086a92-e877-4d67-b76d-9eccb45957e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.268590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b086a92-e877-4d67-b76d-9eccb45957e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.269318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b086a92-e877-4d67-b76d-9eccb45957e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:26.271303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b086a92-e877-4d67-b76d-9eccb45957e4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:26.295694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 945/1900] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:26 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:26.296893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67734851-41d1-4624-afa8-a10ae993ec6c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:26.307300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67734851-41d1-4624-afa8-a10ae993ec6c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:26.312671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1c66b-5628-466b-a518-b818ffbb7024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:26.313341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1c66b-5628-466b-a518-b818ffbb7024 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.313619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1c66b-5628-466b-a518-b818ffbb7024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.313891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca1c66b-5628-466b-a518-b818ffbb7024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.314631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1c66b-5628-466b-a518-b818ffbb7024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:26.316234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca1c66b-5628-466b-a518-b818ffbb7024 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:26.327828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 956/1901] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:26.332173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b086a92-e877-4d67-b76d-9eccb45957e4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:26.342515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b086a92-e877-4d67-b76d-9eccb45957e4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:26.347038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f21c91-4839-4cc7-acfc-582518312928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:26.347958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f21c91-4839-4cc7-acfc-582518312928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.348252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f21c91-4839-4cc7-acfc-582518312928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.348510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f21c91-4839-4cc7-acfc-582518312928 admin admin] PATH_INFO: `/v3/projects/64805a329b57439ba30db0e93e3c4774/users/7e15607fb53e4bcbbf681738e4345433/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.366489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f21c91-4839-4cc7-acfc-582518312928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64805a329b57439ba30db0e93e3c4774, user_id=7e15607fb53e4bcbbf681738e4345433, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:26.369214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f21c91-4839-4cc7-acfc-582518312928 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:26.386402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 946/1902] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:26 2022] PUT /identity/v3/projects/64805a329b57439ba30db0e93e3c4774/users/7e15607fb53e4bcbbf681738e4345433/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:26.390371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1c66b-5628-466b-a518-b818ffbb7024 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:26.401113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca1c66b-5628-466b-a518-b818ffbb7024 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:26.473911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d294757-998a-450d-8b25-99d09af999b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:26.474602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294757-998a-450d-8b25-99d09af999b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.474884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294757-998a-450d-8b25-99d09af999b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.475161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d294757-998a-450d-8b25-99d09af999b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.475960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d294757-998a-450d-8b25-99d09af999b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:26.477881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d294757-998a-450d-8b25-99d09af999b7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:26.488691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 957/1903] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:26.492929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f21c91-4839-4cc7-acfc-582518312928 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:26.503517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f21c91-4839-4cc7-acfc-582518312928 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:26.508109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:26.509072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.509280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.509552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] PATH_INFO: `/v3/projects/64805a329b57439ba30db0e93e3c4774/users/7e15607fb53e4bcbbf681738e4345433/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.512073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64805a329b57439ba30db0e93e3c4774, user_id=7e15607fb53e4bcbbf681738e4345433, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:26.514832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:26.527033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 947/1904] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:26 2022] PUT /identity/v3/projects/64805a329b57439ba30db0e93e3c4774/users/7e15607fb53e4bcbbf681738e4345433/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:26.537748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7209166-df7b-4061-958b-5cdc0b411b9f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.538042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7209166-df7b-4061-958b-5cdc0b411b9f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.538316 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7209166-df7b-4061-958b-5cdc0b411b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.566587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f7209166-df7b-4061-958b-5cdc0b411b9f None None] MFA Rules not processed for user `7e15607fb53e4bcbbf681738e4345433`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:26.567964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f7209166-df7b-4061-958b-5cdc0b411b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:26.653351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 958/1905] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:26.665917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:26.669351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:26.744849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed63ee9-39b1-4742-b576-94b3820a3149 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:26.748573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1946b-04a5-4e49-a763-00582b4d15a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:26.749226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1946b-04a5-4e49-a763-00582b4d15a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.749566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1946b-04a5-4e49-a763-00582b4d15a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.749842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1946b-04a5-4e49-a763-00582b4d15a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.761474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1946b-04a5-4e49-a763-00582b4d15a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:26.763763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1946b-04a5-4e49-a763-00582b4d15a1 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:26.784744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 948/1906] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:26 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:26.857211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7209166-df7b-4061-958b-5cdc0b411b9f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:26.857904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f7209166-df7b-4061-958b-5cdc0b411b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:26.870007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7209166-df7b-4061-958b-5cdc0b411b9f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:26.874609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4f0762-998b-47ce-b4b2-c50ce7e4ad74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:26.875680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0762-998b-47ce-b4b2-c50ce7e4ad74 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:26.875680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0762-998b-47ce-b4b2-c50ce7e4ad74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:26.876109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4f0762-998b-47ce-b4b2-c50ce7e4ad74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:26.939030 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:26.944750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0762-998b-47ce-b4b2-c50ce7e4ad74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:26.946380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4f0762-998b-47ce-b4b2-c50ce7e4ad74 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:26.955989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 959/1907] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:30.335878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1946b-04a5-4e49-a763-00582b4d15a1 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:30.397020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1946b-04a5-4e49-a763-00582b4d15a1 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:30.401947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:30.402825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:30.403108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:30.403407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] PATH_INFO: `/v3/users/0419dec18e67414882042dd3e872f5fa` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:30.404613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0419dec18e67414882042dd3e872f5fa)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:30.406615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:30.461084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0419dec18e67414882042dd3e872f5fa'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:30.461802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0419dec18e67414882042dd3e872f5fa'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:30.462419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0419dec18e67414882042dd3e872f5fa'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:30.468095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:30.473104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0419dec18e67414882042dd3e872f5fa'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:30.482141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0419dec18e67414882042dd3e872f5fa'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:30.499803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 949/1908] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:30 2022] DELETE /identity/v3/users/0419dec18e67414882042dd3e872f5fa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:30.504548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a2c0-e5b0-4b1d-8201-8f3cdc82361f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:30.505037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a2c0-e5b0-4b1d-8201-8f3cdc82361f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:30.505126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a2c0-e5b0-4b1d-8201-8f3cdc82361f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:30.522985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b935a2c0-e5b0-4b1d-8201-8f3cdc82361f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:30.524557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b935a2c0-e5b0-4b1d-8201-8f3cdc82361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:30.583231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 960/1909] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:30.593181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:30.594682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:30.654125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838a8f26-149f-4346-841e-3e29312fcfa9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:30.658604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924713-88dd-4779-a7b2-fd43f0879ba0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:30.659272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924713-88dd-4779-a7b2-fd43f0879ba0 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:30.659593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924713-88dd-4779-a7b2-fd43f0879ba0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:30.659884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924713-88dd-4779-a7b2-fd43f0879ba0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:30.670441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924713-88dd-4779-a7b2-fd43f0879ba0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:30.672021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924713-88dd-4779-a7b2-fd43f0879ba0 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:30.682081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 950/1910] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:30.976155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935a2c0-e5b0-4b1d-8201-8f3cdc82361f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:30.977676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b935a2c0-e5b0-4b1d-8201-8f3cdc82361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:30.991796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935a2c0-e5b0-4b1d-8201-8f3cdc82361f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:30.997616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:30.998459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:30.998739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:30.999005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] PATH_INFO: `/v3/projects/435fb0a7764c467889c0c107f9d3455d` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:31.001256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=435fb0a7764c467889c0c107f9d3455d)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:31.002237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:31.056370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '435fb0a7764c467889c0c107f9d3455d', 'request_id': 'req-41416ebc-3647-430e-b01d-d4b485c686d3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.062197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '435fb0a7764c467889c0c107f9d3455d', 'request_id': 'req-41416ebc-3647-430e-b01d-d4b485c686d3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.068426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '435fb0a7764c467889c0c107f9d3455d', 'request_id': 'req-41416ebc-3647-430e-b01d-d4b485c686d3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.083610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] The token cache is being invalidate because project 435fb0a7764c467889c0c107f9d3455d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:31.083897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.084527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.086320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 961/1911] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:30 2022] DELETE /identity/v3/projects/435fb0a7764c467889c0c107f9d3455d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:31.090253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924713-88dd-4779-a7b2-fd43f0879ba0 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:31.100553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924713-88dd-4779-a7b2-fd43f0879ba0 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:31.160989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:31.161950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:31.162229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:31.162500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] PATH_INFO: `/v3/users/7e15607fb53e4bcbbf681738e4345433` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:31.163741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e15607fb53e4bcbbf681738e4345433)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:31.165691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:31.214676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e15607fb53e4bcbbf681738e4345433'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.215248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e15607fb53e4bcbbf681738e4345433'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.215882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e15607fb53e4bcbbf681738e4345433'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.224529 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e15607fb53e4bcbbf681738e4345433'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.231646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5eb50870-97c9-4520-86c2-f9c5da4b7d51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e15607fb53e4bcbbf681738e4345433'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.240507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:31.252764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 951/1912] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:31 2022] DELETE /identity/v3/users/7e15607fb53e4bcbbf681738e4345433 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:31.520209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:31.521897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:31.540417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41416ebc-3647-430e-b01d-d4b485c686d3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:31.599714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:31.600709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:31.601067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:31.601430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] PATH_INFO: `/v3/projects/64805a329b57439ba30db0e93e3c4774` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:31.602825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64805a329b57439ba30db0e93e3c4774)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:31.605262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:31.631780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:31.658917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64805a329b57439ba30db0e93e3c4774', 'request_id': 'req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.664671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64805a329b57439ba30db0e93e3c4774', 'request_id': 'req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.670951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64805a329b57439ba30db0e93e3c4774', 'request_id': 'req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.684890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] The token cache is being invalidate because project 64805a329b57439ba30db0e93e3c4774 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:31.685171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.685772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-19b2282b-71bc-4c1d-a77b-26aeb4ed5b79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:31.687403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 962/1913] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:31 2022] DELETE /identity/v3/projects/64805a329b57439ba30db0e93e3c4774 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:31.692437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c729ac-247e-4e0b-9833-988029e7c215 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:31.692888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c729ac-247e-4e0b-9833-988029e7c215 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:31.692992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c729ac-247e-4e0b-9833-988029e7c215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:31.712108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-96c729ac-247e-4e0b-9833-988029e7c215 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:31.713319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-96c729ac-247e-4e0b-9833-988029e7c215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:31.774267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 952/1914] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:31.782309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59b4e9-5892-486a-a94a-aa60bef57ba2 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:31.782596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59b4e9-5892-486a-a94a-aa60bef57ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:31.782875 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a59b4e9-5892-486a-a94a-aa60bef57ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:31.801490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4a59b4e9-5892-486a-a94a-aa60bef57ba2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:31.802666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4a59b4e9-5892-486a-a94a-aa60bef57ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:31.810664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 963/1915] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:31.815058 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c729ac-247e-4e0b-9833-988029e7c215 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:31.825400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c729ac-247e-4e0b-9833-988029e7c215 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:31.830091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:31.830737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:31.831014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:31.831288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:31.832038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:31.832341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:31.833941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:31.846252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 953/1916] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:14:31.850423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59b4e9-5892-486a-a94a-aa60bef57ba2 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:31.859999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a59b4e9-5892-486a-a94a-aa60bef57ba2 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:31.865252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e655804-d922-4dba-af4a-a5bb835d9285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:31.865899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e655804-d922-4dba-af4a-a5bb835d9285 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:31.866181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e655804-d922-4dba-af4a-a5bb835d9285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:31.866425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e655804-d922-4dba-af4a-a5bb835d9285 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:31.867151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e655804-d922-4dba-af4a-a5bb835d9285 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:31.869088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e655804-d922-4dba-af4a-a5bb835d9285 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:31.897425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 964/1917] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:31 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:31.901789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:31.912283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375fe5b-21fb-4bd8-9f6e-8d5e62d97b94 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:31.971168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe064-859f-4c9f-aec0-30eb310521bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:31.971891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe064-859f-4c9f-aec0-30eb310521bb admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:31.972175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe064-859f-4c9f-aec0-30eb310521bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:31.972456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfe064-859f-4c9f-aec0-30eb310521bb admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:31.973155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfe064-859f-4c9f-aec0-30eb310521bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:31.975124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbfe064-859f-4c9f-aec0-30eb310521bb admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:31.997703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 954/1918] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:31 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:32.001828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e655804-d922-4dba-af4a-a5bb835d9285 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.012210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e655804-d922-4dba-af4a-a5bb835d9285 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.017293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17028f-768a-46ee-94a9-1c5afcfa4e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.017955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17028f-768a-46ee-94a9-1c5afcfa4e9e admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.018240 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17028f-768a-46ee-94a9-1c5afcfa4e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.018514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e17028f-768a-46ee-94a9-1c5afcfa4e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.019249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17028f-768a-46ee-94a9-1c5afcfa4e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.020885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e17028f-768a-46ee-94a9-1c5afcfa4e9e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.031523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 965/1919] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:32.035442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe064-859f-4c9f-aec0-30eb310521bb admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.045874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfe064-859f-4c9f-aec0-30eb310521bb admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.051543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ce6ca6-c735-4211-9a7b-9363f90de1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.052196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce6ca6-c735-4211-9a7b-9363f90de1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.052465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce6ca6-c735-4211-9a7b-9363f90de1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.052753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ce6ca6-c735-4211-9a7b-9363f90de1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.053489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce6ca6-c735-4211-9a7b-9363f90de1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.055100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ce6ca6-c735-4211-9a7b-9363f90de1d5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.065552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 955/1920] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:32.069841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17028f-768a-46ee-94a9-1c5afcfa4e9e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.084170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e17028f-768a-46ee-94a9-1c5afcfa4e9e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.092626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b974-ac42-4961-9de8-2ccc423fcfc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.093934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b974-ac42-4961-9de8-2ccc423fcfc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.094206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b974-ac42-4961-9de8-2ccc423fcfc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.094491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f833b974-ac42-4961-9de8-2ccc423fcfc7 admin admin] PATH_INFO: `/v3/projects/17ee0eb503fe456db2c806f85d8af2d4/users/89cd44caab574338b2111371ee105dae/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.110098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833b974-ac42-4961-9de8-2ccc423fcfc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ee0eb503fe456db2c806f85d8af2d4, user_id=89cd44caab574338b2111371ee105dae, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.112806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f833b974-ac42-4961-9de8-2ccc423fcfc7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.127609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 966/1921] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:32 2022] PUT /identity/v3/projects/17ee0eb503fe456db2c806f85d8af2d4/users/89cd44caab574338b2111371ee105dae/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:32.144040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199e6af-f560-4646-8e6e-4f767acf3c6c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.144330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199e6af-f560-4646-8e6e-4f767acf3c6c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.144609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199e6af-f560-4646-8e6e-4f767acf3c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.173454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-1199e6af-f560-4646-8e6e-4f767acf3c6c None None] MFA Rules not processed for user `89cd44caab574338b2111371ee105dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:32.174672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1199e6af-f560-4646-8e6e-4f767acf3c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:32.220555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:32.243028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 956/1922] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:32.247531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b974-ac42-4961-9de8-2ccc423fcfc7 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.361689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f833b974-ac42-4961-9de8-2ccc423fcfc7 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.369753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3b883-c90e-4bee-9a51-0f92ef4365e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.370406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3b883-c90e-4bee-9a51-0f92ef4365e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.370915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3b883-c90e-4bee-9a51-0f92ef4365e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.371197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d3b883-c90e-4bee-9a51-0f92ef4365e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.372158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3b883-c90e-4bee-9a51-0f92ef4365e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.375366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d3b883-c90e-4bee-9a51-0f92ef4365e3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.413848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199e6af-f560-4646-8e6e-4f767acf3c6c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.414133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 967/1923] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:32 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:32.443028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199e6af-f560-4646-8e6e-4f767acf3c6c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.544110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62d775-d477-40fb-ac47-8ef6ca7a1377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.545131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62d775-d477-40fb-ac47-8ef6ca7a1377 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.545763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62d775-d477-40fb-ac47-8ef6ca7a1377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.546459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db62d775-d477-40fb-ac47-8ef6ca7a1377 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.547609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62d775-d477-40fb-ac47-8ef6ca7a1377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.551058 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db62d775-d477-40fb-ac47-8ef6ca7a1377 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.583595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 957/1924] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:32 2022] POST /identity/v3/users => generated 296 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:32.587644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3b883-c90e-4bee-9a51-0f92ef4365e3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.600280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d3b883-c90e-4bee-9a51-0f92ef4365e3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.605732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895d05d-e46a-48b2-a9ab-00bc8f20ce5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.606496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895d05d-e46a-48b2-a9ab-00bc8f20ce5d admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.607215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895d05d-e46a-48b2-a9ab-00bc8f20ce5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.607215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895d05d-e46a-48b2-a9ab-00bc8f20ce5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.607932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895d05d-e46a-48b2-a9ab-00bc8f20ce5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.609653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895d05d-e46a-48b2-a9ab-00bc8f20ce5d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.623310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 968/1925] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:32.625510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62d775-d477-40fb-ac47-8ef6ca7a1377 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.635624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db62d775-d477-40fb-ac47-8ef6ca7a1377 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.640326 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83caf0c-ceb1-42be-9b22-05c8f7d5548f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.641244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83caf0c-ceb1-42be-9b22-05c8f7d5548f admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.641527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83caf0c-ceb1-42be-9b22-05c8f7d5548f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.641803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83caf0c-ceb1-42be-9b22-05c8f7d5548f admin admin] PATH_INFO: `/v3/projects/ac346fa66d6949859726a9446b132ca0/users/3794bc0958d74e69a95e00188fce1169/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.657398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83caf0c-ceb1-42be-9b22-05c8f7d5548f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac346fa66d6949859726a9446b132ca0, user_id=3794bc0958d74e69a95e00188fce1169, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.660152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83caf0c-ceb1-42be-9b22-05c8f7d5548f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.675654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 958/1926] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:32 2022] PUT /identity/v3/projects/ac346fa66d6949859726a9446b132ca0/users/3794bc0958d74e69a95e00188fce1169/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:32.680087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895d05d-e46a-48b2-a9ab-00bc8f20ce5d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.691003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895d05d-e46a-48b2-a9ab-00bc8f20ce5d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.710187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:32.759768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a0d2a-3d49-4f7e-b5ae-6cc16a37aabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.761635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0d2a-3d49-4f7e-b5ae-6cc16a37aabf admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.761986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0d2a-3d49-4f7e-b5ae-6cc16a37aabf admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.762341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a0d2a-3d49-4f7e-b5ae-6cc16a37aabf admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.763301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a0d2a-3d49-4f7e-b5ae-6cc16a37aabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.768691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a0d2a-3d49-4f7e-b5ae-6cc16a37aabf admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.785894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 969/1927] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:32.790357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83caf0c-ceb1-42be-9b22-05c8f7d5548f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.801710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83caf0c-ceb1-42be-9b22-05c8f7d5548f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:32.814915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:32.815861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.816147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.816421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] PATH_INFO: `/v3/projects/ac346fa66d6949859726a9446b132ca0/users/3794bc0958d74e69a95e00188fce1169/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.818566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac346fa66d6949859726a9446b132ca0, user_id=3794bc0958d74e69a95e00188fce1169, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:32.821262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:32.839419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 959/1928] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:32 2022] PUT /identity/v3/projects/ac346fa66d6949859726a9446b132ca0/users/3794bc0958d74e69a95e00188fce1169/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:32.843302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052bbc7-e268-40ee-9d3d-fa0c284a7d55 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:32.843876 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052bbc7-e268-40ee-9d3d-fa0c284a7d55 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:32.844291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6052bbc7-e268-40ee-9d3d-fa0c284a7d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:32.884965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6052bbc7-e268-40ee-9d3d-fa0c284a7d55 None None] MFA Rules not processed for user `3794bc0958d74e69a95e00188fce1169`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:32.886186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6052bbc7-e268-40ee-9d3d-fa0c284a7d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:32.967918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 970/1929] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:32 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:32.987556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:32.991199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:33.075540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad806ea8-9e03-4bdc-a0f2-6d7b9185d4f3 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:33.079640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c8301-4401-4c84-822c-80ee94693a3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:33.080307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c8301-4401-4c84-822c-80ee94693a3a service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:33.080615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c8301-4401-4c84-822c-80ee94693a3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:33.081545 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4c8301-4401-4c84-822c-80ee94693a3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:33.095136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c8301-4401-4c84-822c-80ee94693a3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:33.097081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4c8301-4401-4c84-822c-80ee94693a3a service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:33.109048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 960/1930] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:32 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:33.564279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052bbc7-e268-40ee-9d3d-fa0c284a7d55 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:33.576963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6052bbc7-e268-40ee-9d3d-fa0c284a7d55 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:33.584354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3794bc0958d74e69a95e00188fce1169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac346fa66d6949859726a9446b132ca0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:33.585325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:33.585928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:33.586352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:33.587475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:33.587944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:33.590778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:33.612938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 971/1931] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:14:33.617935 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c8301-4401-4c84-822c-80ee94693a3a service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:33.636418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4c8301-4401-4c84-822c-80ee94693a3a service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:33.644821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8138dac-188e-443d-8a75-396738ea76a6 tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3794bc0958d74e69a95e00188fce1169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac346fa66d6949859726a9446b132ca0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:33.646112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8138dac-188e-443d-8a75-396738ea76a6 tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:33.646393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8138dac-188e-443d-8a75-396738ea76a6 tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:33.646671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8138dac-188e-443d-8a75-396738ea76a6 tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:33.647413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8138dac-188e-443d-8a75-396738ea76a6 tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:33.649694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8138dac-188e-443d-8a75-396738ea76a6 tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:33.680212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 961/1932] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:33 2022] POST /identity/v3/projects => generated 353 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:33.681600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:33.692150 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8c8c1-47cc-4818-b10e-76023ce00b1d tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:33.751748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3794bc0958d74e69a95e00188fce1169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac346fa66d6949859726a9446b132ca0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:33.752461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:33.752760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:33.753018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:33.753774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:33.754083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:33.755752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:33.774124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 972/1933] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:14:33.829816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8138dac-188e-443d-8a75-396738ea76a6 tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:33.841081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8138dac-188e-443d-8a75-396738ea76a6 tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:33.845923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3794bc0958d74e69a95e00188fce1169', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ac346fa66d6949859726a9446b132ca0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:33.846925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:33.847215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:33.847884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] PATH_INFO: `/v3/projects/89cf4255f0e64b4fbd6d9d3cf7849fd1` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:33.849125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Authorizing `identity:delete_project(project_id=89cf4255f0e64b4fbd6d9d3cf7849fd1)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:33.851087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:33.868335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:33.911638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89cf4255f0e64b4fbd6d9d3cf7849fd1', 'request_id': 'req-b42759a1-1563-4d65-b168-ce7dd6d7979e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:33.925305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89cf4255f0e64b4fbd6d9d3cf7849fd1', 'request_id': 'req-b42759a1-1563-4d65-b168-ce7dd6d7979e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:33.932050 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89cf4255f0e64b4fbd6d9d3cf7849fd1', 'request_id': 'req-b42759a1-1563-4d65-b168-ce7dd6d7979e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:33.944491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] The token cache is being invalidate because project 89cf4255f0e64b4fbd6d9d3cf7849fd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:33.944776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:33.945459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b42759a1-1563-4d65-b168-ce7dd6d7979e tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:33.947294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 962/1934] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:33 2022] DELETE /identity/v3/projects/89cf4255f0e64b4fbd6d9d3cf7849fd1 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:34.165930 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:34.167878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:34.193979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:34.233538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63294ec4-5f37-49a8-9a51-11840d37192a tempest-VolumeQuotaClassesTest-633026734 tempest-VolumeQuotaClassesTest-633026734-project-admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:34.289480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:34.290442 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:34.290806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:34.291145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] PATH_INFO: `/v3/users/89cd44caab574338b2111371ee105dae` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:34.292395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89cd44caab574338b2111371ee105dae)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:34.294307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:34.343116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89cd44caab574338b2111371ee105dae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:34.350653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89cd44caab574338b2111371ee105dae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:34.351504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89cd44caab574338b2111371ee105dae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:34.360042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89cd44caab574338b2111371ee105dae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:34.360719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89cd44caab574338b2111371ee105dae'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:34.378747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 973/1935] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:34 2022] DELETE /identity/v3/users/89cd44caab574338b2111371ee105dae => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:34.383463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caed75e-f455-4258-81b6-7b54231f1af4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:34.383835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caed75e-f455-4258-81b6-7b54231f1af4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:34.384173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caed75e-f455-4258-81b6-7b54231f1af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:34.402498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-1caed75e-f455-4258-81b6-7b54231f1af4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:34.403817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1caed75e-f455-4258-81b6-7b54231f1af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:34.463352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 963/1936] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:34.473441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:34.474978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:34.531922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:34.536094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb38999-d3fa-474b-8a4e-9e4b0ea733a3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:34.539871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf6a7f-5fae-4840-8409-00e7e6178e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:34.540536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6a7f-5fae-4840-8409-00e7e6178e80 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:34.540816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6a7f-5fae-4840-8409-00e7e6178e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:34.541100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf6a7f-5fae-4840-8409-00e7e6178e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:34.553803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf6a7f-5fae-4840-8409-00e7e6178e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:34.558101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acf6a7f-5fae-4840-8409-00e7e6178e80 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:34.566727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 974/1937] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:34.990729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caed75e-f455-4258-81b6-7b54231f1af4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:34.992232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1caed75e-f455-4258-81b6-7b54231f1af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:35.012796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caed75e-f455-4258-81b6-7b54231f1af4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:35.019468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:35.021393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:35.021746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:35.022081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] PATH_INFO: `/v3/projects/17ee0eb503fe456db2c806f85d8af2d4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:35.023678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ee0eb503fe456db2c806f85d8af2d4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:35.026459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:35.095612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ee0eb503fe456db2c806f85d8af2d4', 'request_id': 'req-7d993525-ba05-4aec-964e-c7a7e137652b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.108436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ee0eb503fe456db2c806f85d8af2d4', 'request_id': 'req-7d993525-ba05-4aec-964e-c7a7e137652b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.115091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ee0eb503fe456db2c806f85d8af2d4', 'request_id': 'req-7d993525-ba05-4aec-964e-c7a7e137652b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.127290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] The token cache is being invalidate because project 17ee0eb503fe456db2c806f85d8af2d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:35.127650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.128474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.130562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 964/1938] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:34 2022] DELETE /identity/v3/projects/17ee0eb503fe456db2c806f85d8af2d4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:35.134516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf6a7f-5fae-4840-8409-00e7e6178e80 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:35.145619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf6a7f-5fae-4840-8409-00e7e6178e80 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:35.209923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:35.211054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:35.211364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:35.211642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] PATH_INFO: `/v3/users/3794bc0958d74e69a95e00188fce1169` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:35.212854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3794bc0958d74e69a95e00188fce1169)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:35.214869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:35.262587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3794bc0958d74e69a95e00188fce1169'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.269595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3794bc0958d74e69a95e00188fce1169'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.270313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3794bc0958d74e69a95e00188fce1169'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.280937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3794bc0958d74e69a95e00188fce1169'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.281562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-930b1495-e7bb-4064-8756-7b55b7bac32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3794bc0958d74e69a95e00188fce1169'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.299996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 975/1939] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:35 2022] DELETE /identity/v3/users/3794bc0958d74e69a95e00188fce1169 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:35.708714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:35.710166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:35.739586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d993525-ba05-4aec-964e-c7a7e137652b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:35.814253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:35.859175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:35.860369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:35.860891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:35.861448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] PATH_INFO: `/v3/projects/ac346fa66d6949859726a9446b132ca0` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:35.862858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac346fa66d6949859726a9446b132ca0)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:35.864900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:35.915381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac346fa66d6949859726a9446b132ca0', 'request_id': 'req-f96bc14f-6714-43a8-8a79-d0614ec67113'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.939550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac346fa66d6949859726a9446b132ca0', 'request_id': 'req-f96bc14f-6714-43a8-8a79-d0614ec67113'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:35.969875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac346fa66d6949859726a9446b132ca0', 'request_id': 'req-f96bc14f-6714-43a8-8a79-d0614ec67113'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:36.001458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] The token cache is being invalidate because project ac346fa66d6949859726a9446b132ca0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:36.002845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:36.004380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96bc14f-6714-43a8-8a79-d0614ec67113 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:36.006973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 965/1940] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:35 2022] DELETE /identity/v3/projects/ac346fa66d6949859726a9446b132ca0 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:36.011433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33931ced-ee6f-4fd5-917d-da2e49fc4fdf None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.011727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33931ced-ee6f-4fd5-917d-da2e49fc4fdf None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.012008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33931ced-ee6f-4fd5-917d-da2e49fc4fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.050643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-33931ced-ee6f-4fd5-917d-da2e49fc4fdf None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:36.051897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-33931ced-ee6f-4fd5-917d-da2e49fc4fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:36.148156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 976/1941] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:36.151917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d354fa-16f5-4f40-9858-ce3fa117acda None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.152706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d354fa-16f5-4f40-9858-ce3fa117acda None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.153071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d354fa-16f5-4f40-9858-ce3fa117acda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.216673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f1d354fa-16f5-4f40-9858-ce3fa117acda None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:36.219099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f1d354fa-16f5-4f40-9858-ce3fa117acda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:36.258509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 966/1942] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:36.263014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33931ced-ee6f-4fd5-917d-da2e49fc4fdf None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:36.278557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33931ced-ee6f-4fd5-917d-da2e49fc4fdf None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:36.292372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:36.293386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.293849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.294271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.295563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:36.297601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:36.299482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:36.318790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:36.334485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 977/1943] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:14:36.334844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d354fa-16f5-4f40-9858-ce3fa117acda None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:36.348473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d354fa-16f5-4f40-9858-ce3fa117acda None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:36.361492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b459120-6212-4bd4-bc23-762c4fad6684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:36.362370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459120-6212-4bd4-bc23-762c4fad6684 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.362803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459120-6212-4bd4-bc23-762c4fad6684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.363217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b459120-6212-4bd4-bc23-762c4fad6684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.364084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b459120-6212-4bd4-bc23-762c4fad6684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:36.366194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b459120-6212-4bd4-bc23-762c4fad6684 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:36.433290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 967/1944] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:36 2022] POST /identity/v3/projects => generated 401 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:36.435766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:36.452511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbaa58b-90ff-40e1-90d0-2f5e61979f76 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:36.553704 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f0480-44d7-46b0-a163-437c0a60c7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:36.554219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f0480-44d7-46b0-a163-437c0a60c7dc admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.554503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f0480-44d7-46b0-a163-437c0a60c7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.554775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f0480-44d7-46b0-a163-437c0a60c7dc admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.556164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f0480-44d7-46b0-a163-437c0a60c7dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:36.558447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3f0480-44d7-46b0-a163-437c0a60c7dc admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:36.589672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 978/1945] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:36 2022] POST /identity/v3/users => generated 301 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:36.591301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b459120-6212-4bd4-bc23-762c4fad6684 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:36.603695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b459120-6212-4bd4-bc23-762c4fad6684 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:36.656982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee8483-8ff0-4f65-8c54-17d07f663632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:36.657747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee8483-8ff0-4f65-8c54-17d07f663632 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.658028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee8483-8ff0-4f65-8c54-17d07f663632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.658306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee8483-8ff0-4f65-8c54-17d07f663632 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.659059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee8483-8ff0-4f65-8c54-17d07f663632 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:36.666379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ee8483-8ff0-4f65-8c54-17d07f663632 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:36.684176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 968/1946] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:36.688290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f0480-44d7-46b0-a163-437c0a60c7dc admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:36.701522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f0480-44d7-46b0-a163-437c0a60c7dc admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:36.711246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c4115f-9361-48b6-b44f-ba1b28b703c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:36.711964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4115f-9361-48b6-b44f-ba1b28b703c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.712251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4115f-9361-48b6-b44f-ba1b28b703c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.712642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c4115f-9361-48b6-b44f-ba1b28b703c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.713399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4115f-9361-48b6-b44f-ba1b28b703c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:36.716616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c4115f-9361-48b6-b44f-ba1b28b703c9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:36.731302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 979/1947] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:36.735451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee8483-8ff0-4f65-8c54-17d07f663632 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:36.747665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ee8483-8ff0-4f65-8c54-17d07f663632 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:36.753132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccaa6e-7422-4145-ab29-3b75ea48e106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:36.754042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccaa6e-7422-4145-ab29-3b75ea48e106 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.754381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccaa6e-7422-4145-ab29-3b75ea48e106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.754665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ccaa6e-7422-4145-ab29-3b75ea48e106 admin admin] PATH_INFO: `/v3/projects/582d399fa91143ffbb71776f28fadf61/users/457080aee82643eeb50f506c17a2398a/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.794630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccaa6e-7422-4145-ab29-3b75ea48e106 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582d399fa91143ffbb71776f28fadf61, user_id=457080aee82643eeb50f506c17a2398a, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:36.798336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ccaa6e-7422-4145-ab29-3b75ea48e106 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:36.823944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 969/1948] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:36 2022] PUT /identity/v3/projects/582d399fa91143ffbb71776f28fadf61/users/457080aee82643eeb50f506c17a2398a/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:36.834486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056779d8-427e-4826-a678-70d8394b5a1f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:36.835764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056779d8-427e-4826-a678-70d8394b5a1f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:36.837096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056779d8-427e-4826-a678-70d8394b5a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:36.894271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-056779d8-427e-4826-a678-70d8394b5a1f None None] MFA Rules not processed for user `457080aee82643eeb50f506c17a2398a`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:36.895782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-056779d8-427e-4826-a678-70d8394b5a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:36.974783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 980/1949] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:36.979335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccaa6e-7422-4145-ab29-3b75ea48e106 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:37.040205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:37.055206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ccaa6e-7422-4145-ab29-3b75ea48e106 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:37.061624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278bab6-d9d1-4b4e-87f8-654ef8e7ccc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:37.062864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278bab6-d9d1-4b4e-87f8-654ef8e7ccc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.062864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278bab6-d9d1-4b4e-87f8-654ef8e7ccc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.063385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278bab6-d9d1-4b4e-87f8-654ef8e7ccc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.065764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278bab6-d9d1-4b4e-87f8-654ef8e7ccc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:37.073251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278bab6-d9d1-4b4e-87f8-654ef8e7ccc5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:37.103643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 970/1950] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:36 2022] POST /identity/v3/projects => generated 399 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:37.108343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056779d8-427e-4826-a678-70d8394b5a1f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:37.120245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056779d8-427e-4826-a678-70d8394b5a1f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:37.196957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf414c4-e7a8-470c-b097-a829e3de2a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:37.197627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf414c4-e7a8-470c-b097-a829e3de2a3d admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.197812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf414c4-e7a8-470c-b097-a829e3de2a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.198084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf414c4-e7a8-470c-b097-a829e3de2a3d admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.198868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf414c4-e7a8-470c-b097-a829e3de2a3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:37.201593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf414c4-e7a8-470c-b097-a829e3de2a3d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:37.212102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:37.229369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 981/1951] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:37 2022] POST /identity/v3/users => generated 306 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:37.230642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278bab6-d9d1-4b4e-87f8-654ef8e7ccc5 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:37.241567 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278bab6-d9d1-4b4e-87f8-654ef8e7ccc5 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:37.246381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb8443-9175-4791-8a2e-2e65dc8d2436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:37.247059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb8443-9175-4791-8a2e-2e65dc8d2436 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.247379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb8443-9175-4791-8a2e-2e65dc8d2436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.247658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb8443-9175-4791-8a2e-2e65dc8d2436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.248411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb8443-9175-4791-8a2e-2e65dc8d2436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:37.250131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bb8443-9175-4791-8a2e-2e65dc8d2436 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:37.261293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 971/1952] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:37.265399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf414c4-e7a8-470c-b097-a829e3de2a3d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:37.276329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf414c4-e7a8-470c-b097-a829e3de2a3d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:37.281307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b6872-ab6f-4c72-87c5-63eecddc0928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:37.282360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b6872-ab6f-4c72-87c5-63eecddc0928 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.282820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b6872-ab6f-4c72-87c5-63eecddc0928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.283139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b6872-ab6f-4c72-87c5-63eecddc0928 admin admin] PATH_INFO: `/v3/projects/c413c25f8045480a80a42fdd30ae1b60/users/2498826f95cd4cb39a5f4e673eb92c9c/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.297796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b6872-ab6f-4c72-87c5-63eecddc0928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c413c25f8045480a80a42fdd30ae1b60, user_id=2498826f95cd4cb39a5f4e673eb92c9c, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:37.300559 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b6872-ab6f-4c72-87c5-63eecddc0928 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:37.316239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 982/1953] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:37 2022] PUT /identity/v3/projects/c413c25f8045480a80a42fdd30ae1b60/users/2498826f95cd4cb39a5f4e673eb92c9c/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:37.320256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb8443-9175-4791-8a2e-2e65dc8d2436 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:37.330673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb8443-9175-4791-8a2e-2e65dc8d2436 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:37.389161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d821f96d-a5bf-4c47-bd3d-86464df6bc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:37.389841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821f96d-a5bf-4c47-bd3d-86464df6bc89 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.390161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821f96d-a5bf-4c47-bd3d-86464df6bc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.390427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d821f96d-a5bf-4c47-bd3d-86464df6bc89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.391196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821f96d-a5bf-4c47-bd3d-86464df6bc89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:37.392828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d821f96d-a5bf-4c47-bd3d-86464df6bc89 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:37.403095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 972/1954] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:37.407675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b6872-ab6f-4c72-87c5-63eecddc0928 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:37.418211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b6872-ab6f-4c72-87c5-63eecddc0928 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:37.423486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:37.424401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.424677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.424949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] PATH_INFO: `/v3/projects/c413c25f8045480a80a42fdd30ae1b60/users/2498826f95cd4cb39a5f4e673eb92c9c/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.427627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c413c25f8045480a80a42fdd30ae1b60, user_id=2498826f95cd4cb39a5f4e673eb92c9c, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:37.430260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:37.440985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 983/1955] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:37 2022] PUT /identity/v3/projects/c413c25f8045480a80a42fdd30ae1b60/users/2498826f95cd4cb39a5f4e673eb92c9c/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:37.451276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385defde-0c00-46c8-84ee-42045ba71fdd None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.451546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385defde-0c00-46c8-84ee-42045ba71fdd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.451829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385defde-0c00-46c8-84ee-42045ba71fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.480628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-385defde-0c00-46c8-84ee-42045ba71fdd None None] MFA Rules not processed for user `2498826f95cd4cb39a5f4e673eb92c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:37.482267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-385defde-0c00-46c8-84ee-42045ba71fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:37.553788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 973/1956] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:37 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:37.564759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:37.566479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:37.626555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4248a5-f48a-40f7-84c9-1832853fc0dd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:37.630285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a830b9-8b66-421d-88eb-e47bb71fbbee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:37.630935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a830b9-8b66-421d-88eb-e47bb71fbbee service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.631216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a830b9-8b66-421d-88eb-e47bb71fbbee service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.631523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a830b9-8b66-421d-88eb-e47bb71fbbee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.640481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a830b9-8b66-421d-88eb-e47bb71fbbee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:37.641949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a830b9-8b66-421d-88eb-e47bb71fbbee service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:37.651869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 984/1957] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:37 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:37.750214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385defde-0c00-46c8-84ee-42045ba71fdd None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:37.760795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385defde-0c00-46c8-84ee-42045ba71fdd None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:37.768020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84e421e-25d0-41e1-ae77-c678c5804bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:37.768698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e421e-25d0-41e1-ae77-c678c5804bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:37.768997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e421e-25d0-41e1-ae77-c678c5804bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:37.769255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84e421e-25d0-41e1-ae77-c678c5804bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:37.824760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:37.841204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e421e-25d0-41e1-ae77-c678c5804bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:37.842678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84e421e-25d0-41e1-ae77-c678c5804bb8 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:37.854447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 974/1958] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:43.012016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a830b9-8b66-421d-88eb-e47bb71fbbee service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:43.076237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a830b9-8b66-421d-88eb-e47bb71fbbee service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:43.081210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:43.082079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:43.082357 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:43.082634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] PATH_INFO: `/v3/users/457080aee82643eeb50f506c17a2398a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:43.083922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] RBAC: Authorizing `identity:delete_user(user_id=457080aee82643eeb50f506c17a2398a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:43.085830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:43.133461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457080aee82643eeb50f506c17a2398a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.140464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457080aee82643eeb50f506c17a2398a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.141159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457080aee82643eeb50f506c17a2398a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.149943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457080aee82643eeb50f506c17a2398a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.150638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457080aee82643eeb50f506c17a2398a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.151962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:43.169021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 985/1959] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:43 2022] DELETE /identity/v3/users/457080aee82643eeb50f506c17a2398a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:43.173553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560bef0-a778-4876-b92d-5e6a7ade2aaa None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:43.174015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560bef0-a778-4876-b92d-5e6a7ade2aaa None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:43.174146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2560bef0-a778-4876-b92d-5e6a7ade2aaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:43.193376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-2560bef0-a778-4876-b92d-5e6a7ade2aaa None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:43.194566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2560bef0-a778-4876-b92d-5e6a7ade2aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:43.255598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 975/1960] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:43.266907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:43.268571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:43.331244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a67a85b-8c3c-4a07-9d31-df859a36321d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:43.335939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83378ce4-c3d2-4918-819e-fcc7a165a081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:43.336587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83378ce4-c3d2-4918-819e-fcc7a165a081 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:43.336862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83378ce4-c3d2-4918-819e-fcc7a165a081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:43.337130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83378ce4-c3d2-4918-819e-fcc7a165a081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:43.346492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83378ce4-c3d2-4918-819e-fcc7a165a081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:43.347960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83378ce4-c3d2-4918-819e-fcc7a165a081 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:43.360424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 986/1961] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:43.714324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560bef0-a778-4876-b92d-5e6a7ade2aaa None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:43.717924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2560bef0-a778-4876-b92d-5e6a7ade2aaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:43.752388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2560bef0-a778-4876-b92d-5e6a7ade2aaa None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:43.757561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:43.758570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:43.758909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:43.759348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] PATH_INFO: `/v3/projects/582d399fa91143ffbb71776f28fadf61` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:43.760684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=582d399fa91143ffbb71776f28fadf61)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:43.762790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:43.827610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '582d399fa91143ffbb71776f28fadf61', 'request_id': 'req-13dd4760-bc7c-42da-8d02-a1b768577682'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.837286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '582d399fa91143ffbb71776f28fadf61', 'request_id': 'req-13dd4760-bc7c-42da-8d02-a1b768577682'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.843691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '582d399fa91143ffbb71776f28fadf61', 'request_id': 'req-13dd4760-bc7c-42da-8d02-a1b768577682'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.855666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] The token cache is being invalidate because project 582d399fa91143ffbb71776f28fadf61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:43.855978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.856615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.858181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 976/1962] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:43 2022] DELETE /identity/v3/projects/582d399fa91143ffbb71776f28fadf61 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:43.862346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83378ce4-c3d2-4918-819e-fcc7a165a081 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:43.872286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83378ce4-c3d2-4918-819e-fcc7a165a081 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:43.928539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:43.929420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:43.929705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:43.929983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] PATH_INFO: `/v3/users/2498826f95cd4cb39a5f4e673eb92c9c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:43.931171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2498826f95cd4cb39a5f4e673eb92c9c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:43.933129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:43.981948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2498826f95cd4cb39a5f4e673eb92c9c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.989661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2498826f95cd4cb39a5f4e673eb92c9c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:43.990462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2498826f95cd4cb39a5f4e673eb92c9c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.000895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2498826f95cd4cb39a5f4e673eb92c9c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.001706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2498826f95cd4cb39a5f4e673eb92c9c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.025238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 987/1963] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:43 2022] DELETE /identity/v3/users/2498826f95cd4cb39a5f4e673eb92c9c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:44.183016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:44.184615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:44.198076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd4760-bc7c-42da-8d02-a1b768577682 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:44.261267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:44.262274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.263104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.263104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] PATH_INFO: `/v3/users/90438171f42e4d74bd4e73bfb39ac91d` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.264475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=90438171f42e4d74bd4e73bfb39ac91d)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:44.266534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:44.282340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:44.334740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90438171f42e4d74bd4e73bfb39ac91d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.335471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90438171f42e4d74bd4e73bfb39ac91d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.336085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90438171f42e4d74bd4e73bfb39ac91d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.346538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:44.346924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90438171f42e4d74bd4e73bfb39ac91d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.348119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:44.359409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:44.360669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-52a2bc26-6e62-4b61-97ad-1abce0b5b1e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90438171f42e4d74bd4e73bfb39ac91d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.369391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46849c0-1c75-4dee-ac8c-d0a342e6de2a admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:44.376608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:44.378031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.379190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.379631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] PATH_INFO: `/v3/projects/c413c25f8045480a80a42fdd30ae1b60` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.381257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c413c25f8045480a80a42fdd30ae1b60)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:44.383620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 977/1964] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:44 2022] DELETE /identity/v3/users/90438171f42e4d74bd4e73bfb39ac91d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:44.383919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:44.452351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c413c25f8045480a80a42fdd30ae1b60', 'request_id': 'req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.465309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c413c25f8045480a80a42fdd30ae1b60', 'request_id': 'req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.472569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c413c25f8045480a80a42fdd30ae1b60', 'request_id': 'req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.489385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] The token cache is being invalidate because project c413c25f8045480a80a42fdd30ae1b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:44.489684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.490339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b101bfcc-57f1-47dc-82ef-1ad555a5a60a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.491935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 988/1965] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:44 2022] DELETE /identity/v3/projects/c413c25f8045480a80a42fdd30ae1b60 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:44.496951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8de0f2-ac38-404b-b4b4-81a347c82df8 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.497241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8de0f2-ac38-404b-b4b4-81a347c82df8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.497544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8de0f2-ac38-404b-b4b4-81a347c82df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.518513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-5f8de0f2-ac38-404b-b4b4-81a347c82df8 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:44.519797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5f8de0f2-ac38-404b-b4b4-81a347c82df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:44.591412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 978/1966] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:44.603935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762002fe-8ecf-4f31-873b-714a14929cf4 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.604129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762002fe-8ecf-4f31-873b-714a14929cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.604484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762002fe-8ecf-4f31-873b-714a14929cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.629012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8de0f2-ac38-404b-b4b4-81a347c82df8 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:44.630657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5f8de0f2-ac38-404b-b4b4-81a347c82df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:44.645076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8de0f2-ac38-404b-b4b4-81a347c82df8 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:44.650739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:44.651851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.652199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.652543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] PATH_INFO: `/v3/projects/e84fe8cfef0c4aa3a55dccc8f9aaddc9` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.654017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84fe8cfef0c4aa3a55dccc8f9aaddc9)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:44.656088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:44.666146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-762002fe-8ecf-4f31-873b-714a14929cf4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:44.667947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-762002fe-8ecf-4f31-873b-714a14929cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:44.681904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 989/1967] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:44.686701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762002fe-8ecf-4f31-873b-714a14929cf4 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:44.700075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762002fe-8ecf-4f31-873b-714a14929cf4 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:44.704757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:44.705451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.705730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.706003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.706733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:44.707098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:44.708722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:44.730477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 990/1968] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:14:44.734923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:44.756038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ccbfa7-6c04-4cc0-b6af-649bae8a6e78 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:44.801521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84fe8cfef0c4aa3a55dccc8f9aaddc9', 'request_id': 'req-95a01838-82dc-4575-b822-7d5bfbc798d3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.814630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84fe8cfef0c4aa3a55dccc8f9aaddc9', 'request_id': 'req-95a01838-82dc-4575-b822-7d5bfbc798d3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.822070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84fe8cfef0c4aa3a55dccc8f9aaddc9', 'request_id': 'req-95a01838-82dc-4575-b822-7d5bfbc798d3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.844863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:44.845272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] The token cache is being invalidate because project e84fe8cfef0c4aa3a55dccc8f9aaddc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:44.845564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.845757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.846135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.846477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.846566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-95a01838-82dc-4575-b822-7d5bfbc798d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:44.847368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:44.850688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 979/1969] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:44 2022] DELETE /identity/v3/projects/e84fe8cfef0c4aa3a55dccc8f9aaddc9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:44.850821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:44.853430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad379e3-4e35-41f7-85cd-a0245285c5b4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.853612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad379e3-4e35-41f7-85cd-a0245285c5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.853890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad379e3-4e35-41f7-85cd-a0245285c5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.873940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4ad379e3-4e35-41f7-85cd-a0245285c5b4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:44.875611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4ad379e3-4e35-41f7-85cd-a0245285c5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:44.882375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 991/1970] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:44 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:44.883680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:44.885232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:44.888111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 980/1971] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:44.895522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b890-bba3-4b4c-b573-f00ae3545353 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.895900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b890-bba3-4b4c-b573-f00ae3545353 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.896271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b890-bba3-4b4c-b573-f00ae3545353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.910265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:44.920719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-1f65b890-bba3-4b4c-b573-f00ae3545353 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:44.922484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1f65b890-bba3-4b4c-b573-f00ae3545353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:44.947784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:44.966148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615b520-7303-4595-b332-fb42d12d3e2e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:44.972402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbf2e-0e0b-4922-9988-0a867acf6cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:44.972576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbf2e-0e0b-4922-9988-0a867acf6cb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:44.972729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbf2e-0e0b-4922-9988-0a867acf6cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:44.973010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccbf2e-0e0b-4922-9988-0a867acf6cb1 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:44.973769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccbf2e-0e0b-4922-9988-0a867acf6cb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:44.975825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccbf2e-0e0b-4922-9988-0a867acf6cb1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:44.995347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 981/1972] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:44.999723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65b890-bba3-4b4c-b573-f00ae3545353 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.004027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 992/1973] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:44 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:45.005389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbf2e-0e0b-4922-9988-0a867acf6cb1 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.014289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65b890-bba3-4b4c-b573-f00ae3545353 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.016376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccbf2e-0e0b-4922-9988-0a867acf6cb1 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.021789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a5c5c-a36f-4939-aff0-464e84aaa26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.022258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.022458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a5c5c-a36f-4939-aff0-464e84aaa26b admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.022758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a5c5c-a36f-4939-aff0-464e84aaa26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.023024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a5c5c-a36f-4939-aff0-464e84aaa26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.023155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.023420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.024041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.024137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a5c5c-a36f-4939-aff0-464e84aaa26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.024778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:45.025130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.025745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a5c5c-a36f-4939-aff0-464e84aaa26b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.027015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.036430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 993/1974] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:45.039489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 982/1975] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:14:45.040951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a5c5c-a36f-4939-aff0-464e84aaa26b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.044635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.052668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a5c5c-a36f-4939-aff0-464e84aaa26b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.057771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcb764-ff75-4ac0-87d2-bec362b5d0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.058015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcb764-ff75-4ac0-87d2-bec362b5d0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.058478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcb764-ff75-4ac0-87d2-bec362b5d0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.058752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fcb764-ff75-4ac0-87d2-bec362b5d0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.059740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcb764-ff75-4ac0-87d2-bec362b5d0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.061265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fcb764-ff75-4ac0-87d2-bec362b5d0e4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.065079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec0e5c8-563f-47f3-b282-1aad11d0ce1e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.072010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428dca08-bba7-46b7-bda7-128c6d33c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.074199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dca08-bba7-46b7-bda7-128c6d33c3d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.074199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dca08-bba7-46b7-bda7-128c6d33c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.074199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dca08-bba7-46b7-bda7-128c6d33c3d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.074199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428dca08-bba7-46b7-bda7-128c6d33c3d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.076761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428dca08-bba7-46b7-bda7-128c6d33c3d9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.083451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 994/1976] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:45.086262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcb764-ff75-4ac0-87d2-bec362b5d0e4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.096601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fcb764-ff75-4ac0-87d2-bec362b5d0e4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.101637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca62e5b-f99b-432d-851e-0c6c1b5a8c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.102546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca62e5b-f99b-432d-851e-0c6c1b5a8c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.102818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca62e5b-f99b-432d-851e-0c6c1b5a8c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.103243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca62e5b-f99b-432d-851e-0c6c1b5a8c0b admin admin] PATH_INFO: `/v3/projects/98d3bea8a0a8490b95320500190332a2/users/f2de40a4164f4460a55c8c5b13c34328/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.113450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 983/1977] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:45 2022] POST /identity/v3/projects => generated 385 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:45.117644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428dca08-bba7-46b7-bda7-128c6d33c3d9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.120633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca62e5b-f99b-432d-851e-0c6c1b5a8c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d3bea8a0a8490b95320500190332a2, user_id=f2de40a4164f4460a55c8c5b13c34328, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.123517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca62e5b-f99b-432d-851e-0c6c1b5a8c0b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.142046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 995/1978] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:45 2022] PUT /identity/v3/projects/98d3bea8a0a8490b95320500190332a2/users/f2de40a4164f4460a55c8c5b13c34328/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:45.152918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f1ff0-1539-4089-8d01-8a9f7eea5f15 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.153303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f1ff0-1539-4089-8d01-8a9f7eea5f15 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.153496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f1ff0-1539-4089-8d01-8a9f7eea5f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.187934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-793f1ff0-1539-4089-8d01-8a9f7eea5f15 None None] MFA Rules not processed for user `f2de40a4164f4460a55c8c5b13c34328`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:45.189596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-793f1ff0-1539-4089-8d01-8a9f7eea5f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:45.195946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428dca08-bba7-46b7-bda7-128c6d33c3d9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.261527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c5eb-c60a-4c04-9b81-f4d4c7f0acfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.262589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c5eb-c60a-4c04-9b81-f4d4c7f0acfb admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.262917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c5eb-c60a-4c04-9b81-f4d4c7f0acfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.263194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c5eb-c60a-4c04-9b81-f4d4c7f0acfb admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.264204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c5eb-c60a-4c04-9b81-f4d4c7f0acfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.266253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c5eb-c60a-4c04-9b81-f4d4c7f0acfb admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.266946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 996/1979] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:45.278055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f1ff0-1539-4089-8d01-8a9f7eea5f15 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.280207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-793f1ff0-1539-4089-8d01-8a9f7eea5f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:45.291300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 984/1980] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:45 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:45.292766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c5eb-c60a-4c04-9b81-f4d4c7f0acfb admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.302845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c5eb-c60a-4c04-9b81-f4d4c7f0acfb admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.307555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6f2dd-dcb1-42c5-94c0-a48617f90187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.308210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6f2dd-dcb1-42c5-94c0-a48617f90187 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.308621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6f2dd-dcb1-42c5-94c0-a48617f90187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.308903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce6f2dd-dcb1-42c5-94c0-a48617f90187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.309747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6f2dd-dcb1-42c5-94c0-a48617f90187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.311476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce6f2dd-dcb1-42c5-94c0-a48617f90187 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.314066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:45.323183 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 985/1981] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:45.327319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6f2dd-dcb1-42c5-94c0-a48617f90187 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.337914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce6f2dd-dcb1-42c5-94c0-a48617f90187 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.342587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee33b7-2890-4dfb-91b0-d3d208da7fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.343433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee33b7-2890-4dfb-91b0-d3d208da7fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.343742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee33b7-2890-4dfb-91b0-d3d208da7fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.344092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee33b7-2890-4dfb-91b0-d3d208da7fd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.344872 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee33b7-2890-4dfb-91b0-d3d208da7fd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.346521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee33b7-2890-4dfb-91b0-d3d208da7fd7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.346856 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f1ff0-1539-4089-8d01-8a9f7eea5f15 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.352871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35669847-7130-42ae-a6bb-728c1afae6a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.354441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35669847-7130-42ae-a6bb-728c1afae6a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.355078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35669847-7130-42ae-a6bb-728c1afae6a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.355477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35669847-7130-42ae-a6bb-728c1afae6a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.358226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 986/1982] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:45.362212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee33b7-2890-4dfb-91b0-d3d208da7fd7 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.366760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35669847-7130-42ae-a6bb-728c1afae6a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.368180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35669847-7130-42ae-a6bb-728c1afae6a0 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.372223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee33b7-2890-4dfb-91b0-d3d208da7fd7 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.377110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 997/1983] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:14:45.377865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14243480-e036-4d7f-80a1-86b2459519ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.378810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14243480-e036-4d7f-80a1-86b2459519ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.379092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14243480-e036-4d7f-80a1-86b2459519ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.379400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14243480-e036-4d7f-80a1-86b2459519ee admin admin] PATH_INFO: `/v3/projects/b6adce7ff0fc4f3a92681b82d44628ba/users/127f93f33ed64b189c4ad8e0ef8db690/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.394137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14243480-e036-4d7f-80a1-86b2459519ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6adce7ff0fc4f3a92681b82d44628ba, user_id=127f93f33ed64b189c4ad8e0ef8db690, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.396892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14243480-e036-4d7f-80a1-86b2459519ee admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.411293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 987/1984] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:45 2022] PUT /identity/v3/projects/b6adce7ff0fc4f3a92681b82d44628ba/users/127f93f33ed64b189c4ad8e0ef8db690/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:45.424935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756db81b-fa11-4485-a3f5-9ddcf2d56708 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.425221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756db81b-fa11-4485-a3f5-9ddcf2d56708 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.425643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756db81b-fa11-4485-a3f5-9ddcf2d56708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.453644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-756db81b-fa11-4485-a3f5-9ddcf2d56708 None None] MFA Rules not processed for user `127f93f33ed64b189c4ad8e0ef8db690`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:45.454850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-756db81b-fa11-4485-a3f5-9ddcf2d56708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:45.467726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:45.520881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 998/1985] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:45.525978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14243480-e036-4d7f-80a1-86b2459519ee admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.609734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14243480-e036-4d7f-80a1-86b2459519ee admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.617906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a724273-b49c-4743-aaa8-f1f9bd7465fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.618757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724273-b49c-4743-aaa8-f1f9bd7465fa admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.618984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724273-b49c-4743-aaa8-f1f9bd7465fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.619275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a724273-b49c-4743-aaa8-f1f9bd7465fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.620003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a724273-b49c-4743-aaa8-f1f9bd7465fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.621960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a724273-b49c-4743-aaa8-f1f9bd7465fa admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.675515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 988/1986] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:45 2022] POST /identity/v3/projects => generated 387 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:45.676921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756db81b-fa11-4485-a3f5-9ddcf2d56708 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.692604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756db81b-fa11-4485-a3f5-9ddcf2d56708 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.769832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb2c5e-0f40-4706-a726-35b1feb6bc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.771947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb2c5e-0f40-4706-a726-35b1feb6bc35 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.771947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb2c5e-0f40-4706-a726-35b1feb6bc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.777247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb2c5e-0f40-4706-a726-35b1feb6bc35 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.777247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb2c5e-0f40-4706-a726-35b1feb6bc35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.777247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb2c5e-0f40-4706-a726-35b1feb6bc35 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.800731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 999/1987] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:45 2022] POST /identity/v3/users => generated 300 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:45.806491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a724273-b49c-4743-aaa8-f1f9bd7465fa admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.817377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a724273-b49c-4743-aaa8-f1f9bd7465fa admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.822936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76051c0-74dd-4860-b7fc-6fe1f3e3e909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.823647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76051c0-74dd-4860-b7fc-6fe1f3e3e909 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.823922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76051c0-74dd-4860-b7fc-6fe1f3e3e909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.824195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76051c0-74dd-4860-b7fc-6fe1f3e3e909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.824933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76051c0-74dd-4860-b7fc-6fe1f3e3e909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.826542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76051c0-74dd-4860-b7fc-6fe1f3e3e909 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.839664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 989/1988] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:45.844179 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb2c5e-0f40-4706-a726-35b1feb6bc35 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.854653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb2c5e-0f40-4706-a726-35b1feb6bc35 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.860070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758feb31-a9b7-4627-be5a-881269f58fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.860997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758feb31-a9b7-4627-be5a-881269f58fc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.861319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758feb31-a9b7-4627-be5a-881269f58fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.861730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758feb31-a9b7-4627-be5a-881269f58fc2 admin admin] PATH_INFO: `/v3/projects/27d99232f46846a0a7a03ad737653936/users/c7e8643cdd574f9ea8824fc2d861d878/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.879821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758feb31-a9b7-4627-be5a-881269f58fc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d99232f46846a0a7a03ad737653936, user_id=c7e8643cdd574f9ea8824fc2d861d878, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.882807 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758feb31-a9b7-4627-be5a-881269f58fc2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:45.898009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1000/1989] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:45 2022] PUT /identity/v3/projects/27d99232f46846a0a7a03ad737653936/users/c7e8643cdd574f9ea8824fc2d861d878/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:45.904293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76051c0-74dd-4860-b7fc-6fe1f3e3e909 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:45.917249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76051c0-74dd-4860-b7fc-6fe1f3e3e909 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:45.994180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb998b6-c459-4376-9372-1e286d505b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:45.994891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb998b6-c459-4376-9372-1e286d505b8a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:45.995742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb998b6-c459-4376-9372-1e286d505b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:45.995742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb998b6-c459-4376-9372-1e286d505b8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:45.996207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb998b6-c459-4376-9372-1e286d505b8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:45.997846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb998b6-c459-4376-9372-1e286d505b8a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:46.011405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 990/1990] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:46.012726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758feb31-a9b7-4627-be5a-881269f58fc2 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:46.024956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758feb31-a9b7-4627-be5a-881269f58fc2 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:46.029985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:46.030934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.031218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.031525 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] PATH_INFO: `/v3/projects/27d99232f46846a0a7a03ad737653936/users/c7e8643cdd574f9ea8824fc2d861d878/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.033566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27d99232f46846a0a7a03ad737653936, user_id=c7e8643cdd574f9ea8824fc2d861d878, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:46.036267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:46.049764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1001/1991] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:46 2022] PUT /identity/v3/projects/27d99232f46846a0a7a03ad737653936/users/c7e8643cdd574f9ea8824fc2d861d878/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:46.061426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4feeb5-8328-4e44-a8f1-79ecedcb4457 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.061730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4feeb5-8328-4e44-a8f1-79ecedcb4457 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.062045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4feeb5-8328-4e44-a8f1-79ecedcb4457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.091141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-df4feeb5-8328-4e44-a8f1-79ecedcb4457 None None] MFA Rules not processed for user `c7e8643cdd574f9ea8824fc2d861d878`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:46.092399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-df4feeb5-8328-4e44-a8f1-79ecedcb4457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:46.113514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:46.174961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 991/1992] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:46.187095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:46.189138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:46.253893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9b5fa-74d9-43e9-a88b-4db5c63edb83 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:46.259156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:46.260344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.260849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.261344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.272180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:46.274219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:46.285201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1002/1993] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:46 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:46.317954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4feeb5-8328-4e44-a8f1-79ecedcb4457 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:46.395817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4feeb5-8328-4e44-a8f1-79ecedcb4457 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:46.400346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:46.401381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.401853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.402772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] PATH_INFO: `/v3/users/127f93f33ed64b189c4ad8e0ef8db690` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.403815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127f93f33ed64b189c4ad8e0ef8db690)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:46.406745 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:46.472049 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127f93f33ed64b189c4ad8e0ef8db690'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.474791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127f93f33ed64b189c4ad8e0ef8db690'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.475526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127f93f33ed64b189c4ad8e0ef8db690'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.475952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:46.478220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:46.484331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127f93f33ed64b189c4ad8e0ef8db690'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.491889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a9723d7-ae0f-43d6-9c75-bcd39fc031b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127f93f33ed64b189c4ad8e0ef8db690'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.492951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24655a54-51cd-4288-9dd0-22e2285e2eee service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:46.497394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:46.498255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.498532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.498799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] PATH_INFO: `/v3/users/209ab22afc2d4e908f3b7d1443e656fb` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.500391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=209ab22afc2d4e908f3b7d1443e656fb)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:46.503074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:46.507991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:46.510404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 992/1994] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:46 2022] DELETE /identity/v3/users/127f93f33ed64b189c4ad8e0ef8db690 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:46.515152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e181f3-7b62-433a-91ee-a1f6e5b3ea76 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.515328 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e181f3-7b62-433a-91ee-a1f6e5b3ea76 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.515587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e181f3-7b62-433a-91ee-a1f6e5b3ea76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.536581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-d2e181f3-7b62-433a-91ee-a1f6e5b3ea76 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:46.537789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d2e181f3-7b62-433a-91ee-a1f6e5b3ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:46.558753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '209ab22afc2d4e908f3b7d1443e656fb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.565938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '209ab22afc2d4e908f3b7d1443e656fb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.566628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '209ab22afc2d4e908f3b7d1443e656fb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.572884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:46.575595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '209ab22afc2d4e908f3b7d1443e656fb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.576198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fcca40d1-b3ef-47f5-86a2-b7ab56b8eb3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '209ab22afc2d4e908f3b7d1443e656fb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:46.594393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1003/1995] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:46 2022] DELETE /identity/v3/users/209ab22afc2d4e908f3b7d1443e656fb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:46.599614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1df27-b27e-4590-869d-e82c8151bdd7 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.599965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1df27-b27e-4590-869d-e82c8151bdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.600237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1df27-b27e-4590-869d-e82c8151bdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.602969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 993/1996] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:46.612586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e181f3-7b62-433a-91ee-a1f6e5b3ea76 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:46.614119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d2e181f3-7b62-433a-91ee-a1f6e5b3ea76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:46.620023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-89a1df27-b27e-4590-869d-e82c8151bdd7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:46.621216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-89a1df27-b27e-4590-869d-e82c8151bdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:46.678627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e181f3-7b62-433a-91ee-a1f6e5b3ea76 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:46.686113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1004/1997] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:46.696527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1df27-b27e-4590-869d-e82c8151bdd7 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:46.753413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cca0d02-4b29-446a-b523-898f3a98f5a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:46.754312 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0d02-4b29-446a-b523-898f3a98f5a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.755162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0d02-4b29-446a-b523-898f3a98f5a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.755561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cca0d02-4b29-446a-b523-898f3a98f5a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.761619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1df27-b27e-4590-869d-e82c8151bdd7 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:46.768902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:46.770023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.770784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.771438 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.772237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca0d02-4b29-446a-b523-898f3a98f5a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:46.774124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cca0d02-4b29-446a-b523-898f3a98f5a6 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:46.782196 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:46.783771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:46.784568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 994/1998] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:46.785856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81e229-af11-4c51-bcf4-34b68d6671be None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:46.786151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81e229-af11-4c51-bcf4-34b68d6671be None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:46.786421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81e229-af11-4c51-bcf4-34b68d6671be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:46.794956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1005/1999] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:14:46.804377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4c81e229-af11-4c51-bcf4-34b68d6671be None None] MFA Rules not processed for user `f2de40a4164f4460a55c8c5b13c34328`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:46.805577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4c81e229-af11-4c51-bcf4-34b68d6671be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:46.861424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 995/2000] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:47.397825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:47.399799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:47.414362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528de97f-78bf-481d-bd04-da20b8bf26b7 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:47.415836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81e229-af11-4c51-bcf4-34b68d6671be None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:47.420103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:47.421118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:47.421463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:47.421919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] PATH_INFO: `/v3/projects/b6adce7ff0fc4f3a92681b82d44628ba` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:47.422178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4c81e229-af11-4c51-bcf4-34b68d6671be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:47.423310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6adce7ff0fc4f3a92681b82d44628ba)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:47.425595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:47.452703 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:47.459511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81e229-af11-4c51-bcf4-34b68d6671be None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:47.467247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:47.468178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:47.468436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:47.468715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] PATH_INFO: `/v3/projects/c5c4ca3bacad4a40ba9aa63313d75417` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:47.470818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c4ca3bacad4a40ba9aa63313d75417)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:47.473646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:47.513467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6adce7ff0fc4f3a92681b82d44628ba', 'request_id': 'req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.520573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6adce7ff0fc4f3a92681b82d44628ba', 'request_id': 'req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.527830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6adce7ff0fc4f3a92681b82d44628ba', 'request_id': 'req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.529380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:47.549404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c4ca3bacad4a40ba9aa63313d75417', 'request_id': 'req-1e7db334-7463-46b3-b5bb-00a127b09ef2'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.553190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] The token cache is being invalidate because project b6adce7ff0fc4f3a92681b82d44628ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:47.553408 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.554138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.556534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1006/2001] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:47 2022] DELETE /identity/v3/projects/b6adce7ff0fc4f3a92681b82d44628ba => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:47.560576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:47.562731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c4ca3bacad4a40ba9aa63313d75417', 'request_id': 'req-1e7db334-7463-46b3-b5bb-00a127b09ef2'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.565834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:47.569334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c4ca3bacad4a40ba9aa63313d75417', 'request_id': 'req-1e7db334-7463-46b3-b5bb-00a127b09ef2'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.584063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] The token cache is being invalidate because project c5c4ca3bacad4a40ba9aa63313d75417 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:47.584371 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.585928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.588098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 996/2002] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:47 2022] DELETE /identity/v3/projects/c5c4ca3bacad4a40ba9aa63313d75417 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:47.599981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:47.602773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:47.649504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b2a1e8-16ad-47b4-bbc5-798d5e915683 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:47.683076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7db334-7463-46b3-b5bb-00a127b09ef2 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:47.690729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:47.691807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:47.692378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:47.692705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] PATH_INFO: `/v3/users/f2cca9e348f44dcbb30ea72d98d2b34e` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:47.705375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2cca9e348f44dcbb30ea72d98d2b34e)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:47.705375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:47.724173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:47.725184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:47.725443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:47.725719 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] PATH_INFO: `/v3/users/c7e8643cdd574f9ea8824fc2d861d878` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:47.727510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7e8643cdd574f9ea8824fc2d861d878)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:47.729568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:47.767620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2cca9e348f44dcbb30ea72d98d2b34e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.772996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2cca9e348f44dcbb30ea72d98d2b34e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.775174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2cca9e348f44dcbb30ea72d98d2b34e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.788868 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2cca9e348f44dcbb30ea72d98d2b34e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.799194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7e8643cdd574f9ea8824fc2d861d878'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.805478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2cca9e348f44dcbb30ea72d98d2b34e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.806360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7e8643cdd574f9ea8824fc2d861d878'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.807123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7e8643cdd574f9ea8824fc2d861d878'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.812012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:47.819806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7e8643cdd574f9ea8824fc2d861d878'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.822226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7e8643cdd574f9ea8824fc2d861d878'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:47.830020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 997/2003] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:47 2022] DELETE /identity/v3/users/f2cca9e348f44dcbb30ea72d98d2b34e => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:47.851491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1007/2004] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:47 2022] DELETE /identity/v3/users/c7e8643cdd574f9ea8824fc2d861d878 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:48.165777 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.167522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:48.215067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.217095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:48.249051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:48.295035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5733082d-5125-4218-8f2a-013abc960e9a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.319040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb3cfc-dd0d-439d-8b57-0915b15aa6bf admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.324553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.325887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.326236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.326611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] PATH_INFO: `/v3/projects/9c1d110ff7734fb49aa31a6ad96b69e7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.328724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1d110ff7734fb49aa31a6ad96b69e7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.330351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.377912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.379552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.379908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.380750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] PATH_INFO: `/v3/projects/27d99232f46846a0a7a03ad737653936` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.382951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27d99232f46846a0a7a03ad737653936)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.385096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.412259 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1d110ff7734fb49aa31a6ad96b69e7', 'request_id': 'req-32552545-40f2-4733-b546-e6505ea4c398'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.418793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1d110ff7734fb49aa31a6ad96b69e7', 'request_id': 'req-32552545-40f2-4733-b546-e6505ea4c398'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.425455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1d110ff7734fb49aa31a6ad96b69e7', 'request_id': 'req-32552545-40f2-4733-b546-e6505ea4c398'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.442078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] The token cache is being invalidate because project 9c1d110ff7734fb49aa31a6ad96b69e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:48.442632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.443798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-32552545-40f2-4733-b546-e6505ea4c398 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.447095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27d99232f46846a0a7a03ad737653936', 'request_id': 'req-fc132430-0050-4a37-b0b5-62b620b07252'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.452906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1008/2005] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:48 2022] DELETE /identity/v3/projects/9c1d110ff7734fb49aa31a6ad96b69e7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:48.458358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27d99232f46846a0a7a03ad737653936', 'request_id': 'req-fc132430-0050-4a37-b0b5-62b620b07252'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.460747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a345d-707b-4428-9325-7a00ba651d2e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.461046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a345d-707b-4428-9325-7a00ba651d2e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.461322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a345d-707b-4428-9325-7a00ba651d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.467760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27d99232f46846a0a7a03ad737653936', 'request_id': 'req-fc132430-0050-4a37-b0b5-62b620b07252'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.479481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] The token cache is being invalidate because project 27d99232f46846a0a7a03ad737653936 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:48.479823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.480488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fc132430-0050-4a37-b0b5-62b620b07252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:48.482088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 998/2006] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:48 2022] DELETE /identity/v3/projects/27d99232f46846a0a7a03ad737653936 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:48.488481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-766a345d-707b-4428-9325-7a00ba651d2e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:48.488854 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776d02f-6804-4e0f-b046-49f7c2e33bb2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.489293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776d02f-6804-4e0f-b046-49f7c2e33bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.489719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c776d02f-6804-4e0f-b046-49f7c2e33bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.490471 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-766a345d-707b-4428-9325-7a00ba651d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:48.517684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c776d02f-6804-4e0f-b046-49f7c2e33bb2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:48.519224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c776d02f-6804-4e0f-b046-49f7c2e33bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:48.564401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1009/2007] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:48.585533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd558-4bde-4d4a-9b27-3028935d4e8f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.585896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd558-4bde-4d4a-9b27-3028935d4e8f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.586242 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1bd558-4bde-4d4a-9b27-3028935d4e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.611299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 999/2008] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:48.618010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac88f-5a4f-4c5f-8126-7ee225267bdc None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.618582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac88f-5a4f-4c5f-8126-7ee225267bdc None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.618878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ac88f-5a4f-4c5f-8126-7ee225267bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.623343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:48.632702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5f1bd558-4bde-4d4a-9b27-3028935d4e8f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:48.634724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5f1bd558-4bde-4d4a-9b27-3028935d4e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:48.638610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-da3ac88f-5a4f-4c5f-8126-7ee225267bdc None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:48.639871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-da3ac88f-5a4f-4c5f-8126-7ee225267bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:48.653094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1010/2009] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:48.653291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1000/2010] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:48.656635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bd558-4bde-4d4a-9b27-3028935d4e8f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.657881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ac88f-5a4f-4c5f-8126-7ee225267bdc None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.670652 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ac88f-5a4f-4c5f-8126-7ee225267bdc None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.671806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1bd558-4bde-4d4a-9b27-3028935d4e8f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.677156 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.677332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.677834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.678116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.678216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.678467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.678579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.678899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.679251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:48.679587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.679813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:48.680173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.681208 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.681991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.696572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1001/2011] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:14:48.697507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.703159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1011/2012] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:14:48.703311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:48.704750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.708633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0f17b0-0e8c-4f24-a1a3-2c782fad3828 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.714675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531d707-2397-4762-889a-81fd1275b243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.715142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984ce9d6-b218-4a4f-8321-14ae3f5af2f6 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.715522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d707-2397-4762-889a-81fd1275b243 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.716017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d707-2397-4762-889a-81fd1275b243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.716373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3531d707-2397-4762-889a-81fd1275b243 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.717253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531d707-2397-4762-889a-81fd1275b243 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.719553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3531d707-2397-4762-889a-81fd1275b243 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.720116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89855-c3e0-4d1c-84d8-f3b5412f2cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.720792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89855-c3e0-4d1c-84d8-f3b5412f2cee admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.721072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89855-c3e0-4d1c-84d8-f3b5412f2cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.721345 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c89855-c3e0-4d1c-84d8-f3b5412f2cee admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.722034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c89855-c3e0-4d1c-84d8-f3b5412f2cee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.724053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c89855-c3e0-4d1c-84d8-f3b5412f2cee admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.747794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1002/2013] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:48.753554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1012/2014] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/projects => generated 355 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:48.753554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89855-c3e0-4d1c-84d8-f3b5412f2cee admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.754454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531d707-2397-4762-889a-81fd1275b243 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.817911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c89855-c3e0-4d1c-84d8-f3b5412f2cee admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.818744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3531d707-2397-4762-889a-81fd1275b243 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.822746 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149100ea-a2b2-4355-b5b9-b294249bd455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.823451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149100ea-a2b2-4355-b5b9-b294249bd455 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.823565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b7fbc-1e22-4813-9eba-3aa27fe1b4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.823738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149100ea-a2b2-4355-b5b9-b294249bd455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.824012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149100ea-a2b2-4355-b5b9-b294249bd455 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.824180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b7fbc-1e22-4813-9eba-3aa27fe1b4cf admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.824456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b7fbc-1e22-4813-9eba-3aa27fe1b4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.824705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149100ea-a2b2-4355-b5b9-b294249bd455 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.824811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b7fbc-1e22-4813-9eba-3aa27fe1b4cf admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.825531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b7fbc-1e22-4813-9eba-3aa27fe1b4cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.826621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149100ea-a2b2-4355-b5b9-b294249bd455 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.827503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b7fbc-1e22-4813-9eba-3aa27fe1b4cf admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.848009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1013/2015] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:48.851141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1003/2016] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/users => generated 278 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:48.851972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149100ea-a2b2-4355-b5b9-b294249bd455 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.854911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b7fbc-1e22-4813-9eba-3aa27fe1b4cf admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.862122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149100ea-a2b2-4355-b5b9-b294249bd455 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.864742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b7fbc-1e22-4813-9eba-3aa27fe1b4cf admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.866778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe59a5-d26a-4a71-b7fb-106ac4a67c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.867467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe59a5-d26a-4a71-b7fb-106ac4a67c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.867747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe59a5-d26a-4a71-b7fb-106ac4a67c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.868015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fe59a5-d26a-4a71-b7fb-106ac4a67c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.868740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe59a5-d26a-4a71-b7fb-106ac4a67c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.869969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ebc00-71ae-4623-a775-1308a4d98340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.870325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fe59a5-d26a-4a71-b7fb-106ac4a67c42 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.870614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ebc00-71ae-4623-a775-1308a4d98340 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.870893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ebc00-71ae-4623-a775-1308a4d98340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.871162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ebc00-71ae-4623-a775-1308a4d98340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.871922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ebc00-71ae-4623-a775-1308a4d98340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.873505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ebc00-71ae-4623-a775-1308a4d98340 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.880270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1014/2017] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:48.883645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1004/2018] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:48.884101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe59a5-d26a-4a71-b7fb-106ac4a67c42 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.887416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ebc00-71ae-4623-a775-1308a4d98340 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.893681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fe59a5-d26a-4a71-b7fb-106ac4a67c42 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.897267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ebc00-71ae-4623-a775-1308a4d98340 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.898368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5029fcb-01dd-410f-84c7-22a90c4e94a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.899389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029fcb-01dd-410f-84c7-22a90c4e94a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.899670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029fcb-01dd-410f-84c7-22a90c4e94a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.899925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5029fcb-01dd-410f-84c7-22a90c4e94a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.900756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5029fcb-01dd-410f-84c7-22a90c4e94a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.902322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5029fcb-01dd-410f-84c7-22a90c4e94a9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.903102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b6b2c-1ab2-480c-8f1e-5cc04460ecbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.903792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b6b2c-1ab2-480c-8f1e-5cc04460ecbc admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.904059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b6b2c-1ab2-480c-8f1e-5cc04460ecbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.904329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0b6b2c-1ab2-480c-8f1e-5cc04460ecbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.905069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b6b2c-1ab2-480c-8f1e-5cc04460ecbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.906627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0b6b2c-1ab2-480c-8f1e-5cc04460ecbc admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.913152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1015/2019] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:48.920127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1005/2020] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:48.920287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5029fcb-01dd-410f-84c7-22a90c4e94a9 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.921061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b6b2c-1ab2-480c-8f1e-5cc04460ecbc admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:48.926851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5029fcb-01dd-410f-84c7-22a90c4e94a9 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.930506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0b6b2c-1ab2-480c-8f1e-5cc04460ecbc admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:48.931582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b167083-f2ba-4558-9fa2-4a4217c0d525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.932430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b167083-f2ba-4558-9fa2-4a4217c0d525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.932708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b167083-f2ba-4558-9fa2-4a4217c0d525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.932981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b167083-f2ba-4558-9fa2-4a4217c0d525 admin admin] PATH_INFO: `/v3/projects/36265a64acbe4cfaaeb4613ecaa437b7/users/87d3b4164381427db71fc090f7a48dbf/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.935410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a92e0b-33ed-4966-a21d-a73266d21b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:48.936153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92e0b-33ed-4966-a21d-a73266d21b17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.936456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92e0b-33ed-4966-a21d-a73266d21b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.936706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a92e0b-33ed-4966-a21d-a73266d21b17 admin admin] PATH_INFO: `/v3/projects/d9399282a8ba4f87abe2834702a061b1/users/caec2af73cc0488a89130a2715503bf3/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.948140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b167083-f2ba-4558-9fa2-4a4217c0d525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36265a64acbe4cfaaeb4613ecaa437b7, user_id=87d3b4164381427db71fc090f7a48dbf, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.950872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b167083-f2ba-4558-9fa2-4a4217c0d525 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.951915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a92e0b-33ed-4966-a21d-a73266d21b17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9399282a8ba4f87abe2834702a061b1, user_id=caec2af73cc0488a89130a2715503bf3, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:48.954678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a92e0b-33ed-4966-a21d-a73266d21b17 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:48.964940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1016/2021] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:48 2022] PUT /identity/v3/projects/36265a64acbe4cfaaeb4613ecaa437b7/users/87d3b4164381427db71fc090f7a48dbf/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:48.968400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1006/2022] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:48 2022] PUT /identity/v3/projects/d9399282a8ba4f87abe2834702a061b1/users/caec2af73cc0488a89130a2715503bf3/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:48.976539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce85af-eb17-478d-805e-84b75fc99fa8 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.976833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce85af-eb17-478d-805e-84b75fc99fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.977164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce85af-eb17-478d-805e-84b75fc99fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:48.978707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d1ca-aa8c-4a1b-b4d7-96aee93e6a67 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:48.978995 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d1ca-aa8c-4a1b-b4d7-96aee93e6a67 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:48.979407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6519d1ca-aa8c-4a1b-b4d7-96aee93e6a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.003112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2bce85af-eb17-478d-805e-84b75fc99fa8 None None] MFA Rules not processed for user `87d3b4164381427db71fc090f7a48dbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:49.004368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2bce85af-eb17-478d-805e-84b75fc99fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.007587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6519d1ca-aa8c-4a1b-b4d7-96aee93e6a67 None None] MFA Rules not processed for user `caec2af73cc0488a89130a2715503bf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:49.008757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6519d1ca-aa8c-4a1b-b4d7-96aee93e6a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.063032 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:49.069084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1017/2023] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:49.076552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1007/2024] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:48 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:49.080246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce85af-eb17-478d-805e-84b75fc99fa8 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.081042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519d1ca-aa8c-4a1b-b4d7-96aee93e6a67 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.081595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2bce85af-eb17-478d-805e-84b75fc99fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.093895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:49.138603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce85af-eb17-478d-805e-84b75fc99fa8 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.142309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d9b1-8121-4959-95cc-70d88fe47d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.142980 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d9b1-8121-4959-95cc-70d88fe47d5b service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.143307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d9b1-8121-4959-95cc-70d88fe47d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.143562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d9b1-8121-4959-95cc-70d88fe47d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.145142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6519d1ca-aa8c-4a1b-b4d7-96aee93e6a67 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.150233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee88bc-1266-41a7-8c28-b8421c5b5816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.150933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee88bc-1266-41a7-8c28-b8421c5b5816 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.151220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee88bc-1266-41a7-8c28-b8421c5b5816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.151522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ee88bc-1266-41a7-8c28-b8421c5b5816 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.152229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee88bc-1266-41a7-8c28-b8421c5b5816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.152385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d9b1-8121-4959-95cc-70d88fe47d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.153835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d9b1-8121-4959-95cc-70d88fe47d5b service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.154311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ee88bc-1266-41a7-8c28-b8421c5b5816 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.162653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1018/2025] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:49 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:49.181123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1008/2026] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:49 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:14:49.185557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d9b1-8121-4959-95cc-70d88fe47d5b service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.186545 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee88bc-1266-41a7-8c28-b8421c5b5816 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.196266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d9b1-8121-4959-95cc-70d88fe47d5b service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.247573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ee88bc-1266-41a7-8c28-b8421c5b5816 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.252402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.253284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.253566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.253841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] PATH_INFO: `/v3/users/87d3b4164381427db71fc090f7a48dbf` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.254292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b7b43-6f9b-44c0-bfb8-3cdd7ef9b1b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.254963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b7b43-6f9b-44c0-bfb8-3cdd7ef9b1b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.255064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d3b4164381427db71fc090f7a48dbf)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.255246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b7b43-6f9b-44c0-bfb8-3cdd7ef9b1b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.255552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b7b43-6f9b-44c0-bfb8-3cdd7ef9b1b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.256243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b7b43-6f9b-44c0-bfb8-3cdd7ef9b1b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.256974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.258174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b7b43-6f9b-44c0-bfb8-3cdd7ef9b1b5 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.279431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1019/2027] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:49 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:49.283918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b7b43-6f9b-44c0-bfb8-3cdd7ef9b1b5 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.294158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b7b43-6f9b-44c0-bfb8-3cdd7ef9b1b5 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.299327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.299978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.300260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.300588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.301415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.302854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.306193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d3b4164381427db71fc090f7a48dbf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:49.307030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d3b4164381427db71fc090f7a48dbf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:49.307672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d3b4164381427db71fc090f7a48dbf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:49.312893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1020/2028] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:49.317117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d3b4164381427db71fc090f7a48dbf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:49.317247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.319041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.324660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0b2b6cd2-158f-4561-bd66-a56e3b414aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d3b4164381427db71fc090f7a48dbf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:49.331573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f263107-c676-429d-9ffc-94bec696a8f2 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.336097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7310ea5-e3d2-44e2-aa41-0d68c84af1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.336976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7310ea5-e3d2-44e2-aa41-0d68c84af1cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.337254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7310ea5-e3d2-44e2-aa41-0d68c84af1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.337530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7310ea5-e3d2-44e2-aa41-0d68c84af1cb admin admin] PATH_INFO: `/v3/projects/218ee4bf994b4493840ab19aca28648a/users/f16a1a18b66445829ba4b3efb3ad5153/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.341511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1009/2029] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:14:49 2022] DELETE /identity/v3/users/87d3b4164381427db71fc090f7a48dbf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:49.345999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0127f9-15d6-43f2-956f-22d3b4be1843 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.346771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0127f9-15d6-43f2-956f-22d3b4be1843 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.347053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0127f9-15d6-43f2-956f-22d3b4be1843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.352340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7310ea5-e3d2-44e2-aa41-0d68c84af1cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=218ee4bf994b4493840ab19aca28648a, user_id=f16a1a18b66445829ba4b3efb3ad5153, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.354991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7310ea5-e3d2-44e2-aa41-0d68c84af1cb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.364449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-fe0127f9-15d6-43f2-956f-22d3b4be1843 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:49.365801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fe0127f9-15d6-43f2-956f-22d3b4be1843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.369443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1021/2030] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:49 2022] PUT /identity/v3/projects/218ee4bf994b4493840ab19aca28648a/users/f16a1a18b66445829ba4b3efb3ad5153/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:49.373132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7310ea5-e3d2-44e2-aa41-0d68c84af1cb admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.378240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:49.428722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1010/2031] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:49.429907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:49.435286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7310ea5-e3d2-44e2-aa41-0d68c84af1cb admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.438680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0127f9-15d6-43f2-956f-22d3b4be1843 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.440557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fe0127f9-15d6-43f2-956f-22d3b4be1843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.441347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9110b-6b84-45e0-99eb-09f1898ff4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.442018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9110b-6b84-45e0-99eb-09f1898ff4d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.442305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9110b-6b84-45e0-99eb-09f1898ff4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.442577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9110b-6b84-45e0-99eb-09f1898ff4d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.443369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9110b-6b84-45e0-99eb-09f1898ff4d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.444968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9110b-6b84-45e0-99eb-09f1898ff4d1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.454835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1022/2032] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:14:49.458854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9110b-6b84-45e0-99eb-09f1898ff4d1 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.468079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9110b-6b84-45e0-99eb-09f1898ff4d1 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.472774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382f559-1af9-4f47-b593-bb74b9415890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.473633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f559-1af9-4f47-b593-bb74b9415890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.473910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f559-1af9-4f47-b593-bb74b9415890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.474181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382f559-1af9-4f47-b593-bb74b9415890 admin admin] PATH_INFO: `/v3/projects/218ee4bf994b4493840ab19aca28648a/users/f16a1a18b66445829ba4b3efb3ad5153/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.476468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382f559-1af9-4f47-b593-bb74b9415890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=218ee4bf994b4493840ab19aca28648a, user_id=f16a1a18b66445829ba4b3efb3ad5153, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.479281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382f559-1af9-4f47-b593-bb74b9415890 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.490578 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1023/2033] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:49 2022] PUT /identity/v3/projects/218ee4bf994b4493840ab19aca28648a/users/f16a1a18b66445829ba4b3efb3ad5153/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:14:49.499679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0127f9-15d6-43f2-956f-22d3b4be1843 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.500545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0be42f-c02b-4681-8b85-b35747d9bf01 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.500834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0be42f-c02b-4681-8b85-b35747d9bf01 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.501113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0be42f-c02b-4681-8b85-b35747d9bf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.531439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1c0be42f-c02b-4681-8b85-b35747d9bf01 None None] MFA Rules not processed for user `f16a1a18b66445829ba4b3efb3ad5153`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:49.532776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1c0be42f-c02b-4681-8b85-b35747d9bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.552520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc795-c6e3-4985-adaa-3e3482b330a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.553463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc795-c6e3-4985-adaa-3e3482b330a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.553937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc795-c6e3-4985-adaa-3e3482b330a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.554238 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1bc795-c6e3-4985-adaa-3e3482b330a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.612402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1024/2034] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:49 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:49.624486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0be42f-c02b-4681-8b85-b35747d9bf01 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.626230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1c0be42f-c02b-4681-8b85-b35747d9bf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.636792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc795-c6e3-4985-adaa-3e3482b330a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.638309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1bc795-c6e3-4985-adaa-3e3482b330a4 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.649365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1011/2035] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:49.685702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0be42f-c02b-4681-8b85-b35747d9bf01 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.693326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.694087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.694433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.694715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.708011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.709542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.720390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1025/2036] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:49 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:49.826510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc795-c6e3-4985-adaa-3e3482b330a4 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.837469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1bc795-c6e3-4985-adaa-3e3482b330a4 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:49.841215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b0b315-9a22-4884-87c9-5d9413d8a71a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:49.842020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b315-9a22-4884-87c9-5d9413d8a71a service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:49.842366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b315-9a22-4884-87c9-5d9413d8a71a service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:49.842705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b0b315-9a22-4884-87c9-5d9413d8a71a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:49.844679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-87b0b315-9a22-4884-87c9-5d9413d8a71a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:49.870797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:49.909426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0b315-9a22-4884-87c9-5d9413d8a71a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:49.910882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b0b315-9a22-4884-87c9-5d9413d8a71a service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:49.921485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1012/2037] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:49 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:49.983755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:49.985675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:50.007911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd657d-549c-4f69-b0a6-627875530bc3 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:50.012461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:50.013321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.013597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.013870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] PATH_INFO: `/v3/projects/36265a64acbe4cfaaeb4613ecaa437b7` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.015193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] RBAC: Authorizing `identity:delete_project(project_id=36265a64acbe4cfaaeb4613ecaa437b7)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:50.017258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:50.101905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36265a64acbe4cfaaeb4613ecaa437b7', 'request_id': 'req-2330160f-5948-494b-ac09-236e94703fec'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:50.108909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36265a64acbe4cfaaeb4613ecaa437b7', 'request_id': 'req-2330160f-5948-494b-ac09-236e94703fec'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:50.110783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:50.119331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36265a64acbe4cfaaeb4613ecaa437b7', 'request_id': 'req-2330160f-5948-494b-ac09-236e94703fec'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:50.142178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] The token cache is being invalidate because project 36265a64acbe4cfaaeb4613ecaa437b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:14:50.142715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:50.143414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2330160f-5948-494b-ac09-236e94703fec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:14:50.145019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1026/2038] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:14:49 2022] DELETE /identity/v3/projects/36265a64acbe4cfaaeb4613ecaa437b7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:50.151861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f499ece-c52b-4bbb-aa00-c3d915445fd4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.152161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f499ece-c52b-4bbb-aa00-c3d915445fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.152809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f499ece-c52b-4bbb-aa00-c3d915445fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.172527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9f499ece-c52b-4bbb-aa00-c3d915445fd4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:50.173794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9f499ece-c52b-4bbb-aa00-c3d915445fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:50.235497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1013/2039] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:50.243705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b70dd-db91-41f1-ae15-e93d3c2784da None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.244139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b70dd-db91-41f1-ae15-e93d3c2784da None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.244526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b70dd-db91-41f1-ae15-e93d3c2784da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.265306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6a4b70dd-db91-41f1-ae15-e93d3c2784da None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:50.266517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6a4b70dd-db91-41f1-ae15-e93d3c2784da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:50.275797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1027/2040] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:14:50.280056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f499ece-c52b-4bbb-aa00-c3d915445fd4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:50.290835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f499ece-c52b-4bbb-aa00-c3d915445fd4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:50.295633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:50.296299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.296581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.296853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.297591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:14:50.297896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:50.299736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:50.313405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1014/2041] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:14:50.317747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b70dd-db91-41f1-ae15-e93d3c2784da None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:50.328205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b70dd-db91-41f1-ae15-e93d3c2784da None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:50.332921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33722a-fa90-4151-9915-f14d6d924d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:50.333577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33722a-fa90-4151-9915-f14d6d924d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.333854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33722a-fa90-4151-9915-f14d6d924d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.334122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d33722a-fa90-4151-9915-f14d6d924d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.334820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33722a-fa90-4151-9915-f14d6d924d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:50.336748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d33722a-fa90-4151-9915-f14d6d924d9c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:50.370716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1028/2042] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:14:50 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:50.372243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:50.384194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c24ab-1b3e-4f72-9f03-502dbd0fb617 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:50.446496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db24ca2-82c7-4cf1-a9b5-208909080309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:50.447250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24ca2-82c7-4cf1-a9b5-208909080309 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.447623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24ca2-82c7-4cf1-a9b5-208909080309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.447957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db24ca2-82c7-4cf1-a9b5-208909080309 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.448734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db24ca2-82c7-4cf1-a9b5-208909080309 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:50.450731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db24ca2-82c7-4cf1-a9b5-208909080309 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:50.472261 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1015/2043] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:14:50 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:14:50.476616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33722a-fa90-4151-9915-f14d6d924d9c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:50.487641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d33722a-fa90-4151-9915-f14d6d924d9c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:50.494221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d63cdd-ac78-42b8-813b-d487a1318018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:50.494896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d63cdd-ac78-42b8-813b-d487a1318018 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.495299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d63cdd-ac78-42b8-813b-d487a1318018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.495512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d63cdd-ac78-42b8-813b-d487a1318018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.496280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d63cdd-ac78-42b8-813b-d487a1318018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:50.497991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d63cdd-ac78-42b8-813b-d487a1318018 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:50.510923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1029/2044] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:50.514984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db24ca2-82c7-4cf1-a9b5-208909080309 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:50.525844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db24ca2-82c7-4cf1-a9b5-208909080309 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:50.531515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42484840-4a92-4565-a288-4ab85b38cf1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:50.532311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42484840-4a92-4565-a288-4ab85b38cf1d admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.532720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42484840-4a92-4565-a288-4ab85b38cf1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.533134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42484840-4a92-4565-a288-4ab85b38cf1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.533972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42484840-4a92-4565-a288-4ab85b38cf1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:50.535761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42484840-4a92-4565-a288-4ab85b38cf1d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:50.546699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1016/2045] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:14:50.551158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d63cdd-ac78-42b8-813b-d487a1318018 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:50.563183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d63cdd-ac78-42b8-813b-d487a1318018 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:50.568613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:50.569591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.569934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.570276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] PATH_INFO: `/v3/projects/03179057a85f48688869ec65ba57021a/users/854ec29279834fec97ff68e9cac87413/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.586173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03179057a85f48688869ec65ba57021a, user_id=854ec29279834fec97ff68e9cac87413, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:50.589007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:50.608502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1030/2046] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:14:50 2022] PUT /identity/v3/projects/03179057a85f48688869ec65ba57021a/users/854ec29279834fec97ff68e9cac87413/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:14:50.618783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e636a0e-5da6-492b-9a4a-1b95946daa7a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.619073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e636a0e-5da6-492b-9a4a-1b95946daa7a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.619377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e636a0e-5da6-492b-9a4a-1b95946daa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.648241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6e636a0e-5da6-492b-9a4a-1b95946daa7a None None] MFA Rules not processed for user `854ec29279834fec97ff68e9cac87413`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:50.649760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6e636a0e-5da6-492b-9a4a-1b95946daa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:50.670319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:14:50.744145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1017/2047] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:14:50.752869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:14:50.755046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:50.817632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62f44b5-3192-444e-bb26-7343170cf5b6 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:14:50.821564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaf96c3-ce39-41ef-897b-b39e1ed044d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:14:50.823476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf96c3-ce39-41ef-897b-b39e1ed044d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:50.823880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf96c3-ce39-41ef-897b-b39e1ed044d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:50.824644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaf96c3-ce39-41ef-897b-b39e1ed044d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:50.834248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf96c3-ce39-41ef-897b-b39e1ed044d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:14:50.835728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaf96c3-ce39-41ef-897b-b39e1ed044d9 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:14:50.848766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1031/2048] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:14:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:14:52.225522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4c1e-4a14-4aac-9ef3-0a6f211b3126 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:14:52.226502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4c1e-4a14-4aac-9ef3-0a6f211b3126 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:14:52.226775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc4c1e-4a14-4aac-9ef3-0a6f211b3126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:14:52.245276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a1cc4c1e-4a14-4aac-9ef3-0a6f211b3126 None None] MFA Rules not processed for user `712133b246c94438b94c1f865dca1be9`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:14:52.246522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a1cc4c1e-4a14-4aac-9ef3-0a6f211b3126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:14:52.300267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1018/2049] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:14:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:00.843472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=56,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:02.293808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=44 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:09.113130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00f3f-c25a-4774-b836-d18bcae23012 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:09.113631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00f3f-c25a-4774-b836-d18bcae23012 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:09.113718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c00f3f-c25a-4774-b836-d18bcae23012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:09.133181 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b4c00f3f-c25a-4774-b836-d18bcae23012 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:09.134364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b4c00f3f-c25a-4774-b836-d18bcae23012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:09.199283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1032/2050] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:15:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:09.240759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4c1e-4a14-4aac-9ef3-0a6f211b3126 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:09.242211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a1cc4c1e-4a14-4aac-9ef3-0a6f211b3126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:09.312165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc4c1e-4a14-4aac-9ef3-0a6f211b3126 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:09.317112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b253168-3dad-4b8a-b77e-91c9495031b4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:09.317822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b253168-3dad-4b8a-b77e-91c9495031b4 service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:09.318043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b253168-3dad-4b8a-b77e-91c9495031b4 service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:09.318317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b253168-3dad-4b8a-b77e-91c9495031b4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:09.326218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b253168-3dad-4b8a-b77e-91c9495031b4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:09.327858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b253168-3dad-4b8a-b77e-91c9495031b4 service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:09.335748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1019/2051] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:15:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:15:09.884145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833893c-9b54-4e73-9c13-18c9b6a43b64 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:09.884433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833893c-9b54-4e73-9c13-18c9b6a43b64 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:09.884714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8833893c-9b54-4e73-9c13-18c9b6a43b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:09.903084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8833893c-9b54-4e73-9c13-18c9b6a43b64 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:09.904344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8833893c-9b54-4e73-9c13-18c9b6a43b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:09.913146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1033/2052] 10.222.0.69 () {58 vars in 967 bytes} [Fri May 20 09:15:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:09.929546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b253168-3dad-4b8a-b77e-91c9495031b4 service swift] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:09.941749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b253168-3dad-4b8a-b77e-91c9495031b4 service swift] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:09.956581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581e81e-25c6-4e33-9af4-b94bd3bda30c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:09.957074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581e81e-25c6-4e33-9af4-b94bd3bda30c service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:09.957357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581e81e-25c6-4e33-9af4-b94bd3bda30c service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:09.957698 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c581e81e-25c6-4e33-9af4-b94bd3bda30c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:09.975922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581e81e-25c6-4e33-9af4-b94bd3bda30c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:09.977958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c581e81e-25c6-4e33-9af4-b94bd3bda30c service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:09.990512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1020/2053] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:15:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:15:10.251935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551acc2c-59c8-4799-b70b-a68d05c16e84 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:10.252433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551acc2c-59c8-4799-b70b-a68d05c16e84 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:10.252528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551acc2c-59c8-4799-b70b-a68d05c16e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:10.276664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-551acc2c-59c8-4799-b70b-a68d05c16e84 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:10.276664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-551acc2c-59c8-4799-b70b-a68d05c16e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:10.278864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1034/2054] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:10.291048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581e81e-25c6-4e33-9af4-b94bd3bda30c service swift] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:10.302063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c581e81e-25c6-4e33-9af4-b94bd3bda30c service swift] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:10.307326 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898adf16-f845-4c9d-8574-81cbbd736636 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:10.308057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898adf16-f845-4c9d-8574-81cbbd736636 service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:10.308562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898adf16-f845-4c9d-8574-81cbbd736636 service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:10.308906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898adf16-f845-4c9d-8574-81cbbd736636 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:10.317666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898adf16-f845-4c9d-8574-81cbbd736636 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:10.319371 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898adf16-f845-4c9d-8574-81cbbd736636 service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:10.327679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1021/2055] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:15:10.678729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449193f2-0a0a-4d16-844e-040f9b5232a9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:10.679020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449193f2-0a0a-4d16-844e-040f9b5232a9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:10.679332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449193f2-0a0a-4d16-844e-040f9b5232a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:10.697465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-449193f2-0a0a-4d16-844e-040f9b5232a9 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:10.698879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-449193f2-0a0a-4d16-844e-040f9b5232a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:10.706811 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1035/2056] 10.222.0.69 () {58 vars in 967 bytes} [Fri May 20 09:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:10.719653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898adf16-f845-4c9d-8574-81cbbd736636 service swift] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:10.730893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898adf16-f845-4c9d-8574-81cbbd736636 service swift] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:10.736851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b2095-1102-4348-ab84-df37058caa44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:10.737582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2095-1102-4348-ab84-df37058caa44 service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:10.737899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2095-1102-4348-ab84-df37058caa44 service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:10.738221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b2095-1102-4348-ab84-df37058caa44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:10.748846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b2095-1102-4348-ab84-df37058caa44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:10.750505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b2095-1102-4348-ab84-df37058caa44 service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:10.760651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1022/2057] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:15:10.915501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449193f2-0a0a-4d16-844e-040f9b5232a9 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:10.984116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449193f2-0a0a-4d16-844e-040f9b5232a9 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:10.988795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:10.989734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:10.990016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:10.990343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] PATH_INFO: `/v3/users/854ec29279834fec97ff68e9cac87413` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:10.991664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] RBAC: Authorizing `identity:delete_user(user_id=854ec29279834fec97ff68e9cac87413)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:10.994002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:11.047382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854ec29279834fec97ff68e9cac87413'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.054739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854ec29279834fec97ff68e9cac87413'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.055470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854ec29279834fec97ff68e9cac87413'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.065382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854ec29279834fec97ff68e9cac87413'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.066177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854ec29279834fec97ff68e9cac87413'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.095521 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1036/2058] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:15:10 2022] DELETE /identity/v3/users/854ec29279834fec97ff68e9cac87413 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:11.101098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a4b6-b2ce-4d85-95ad-6bd1e5d0cb96 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:11.101391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a4b6-b2ce-4d85-95ad-6bd1e5d0cb96 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:11.101676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206a4b6-b2ce-4d85-95ad-6bd1e5d0cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:11.124860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f206a4b6-b2ce-4d85-95ad-6bd1e5d0cb96 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:11.126294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f206a4b6-b2ce-4d85-95ad-6bd1e5d0cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:11.205870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1023/2059] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:11.215830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:11.217338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:11.226026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:11.298317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13923aad-d4f2-481c-9ea7-57f5e09c0567 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:11.306996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:11.307820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:11.308206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:11.308686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:11.319296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:11.320652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:11.335116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1037/2060] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:15:11.660356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206a4b6-b2ce-4d85-95ad-6bd1e5d0cb96 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:11.660356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f206a4b6-b2ce-4d85-95ad-6bd1e5d0cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:11.675096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206a4b6-b2ce-4d85-95ad-6bd1e5d0cb96 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:11.680460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:11.681486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:11.681773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:11.682247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] PATH_INFO: `/v3/users/caec2af73cc0488a89130a2715503bf3` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:11.684112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caec2af73cc0488a89130a2715503bf3)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:11.686472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:11.690398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:11.691975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:11.711943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d12c4-fd53-4202-ab60-9aef439a582e service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:11.726328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:11.727218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:11.730041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:11.730314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] PATH_INFO: `/v3/projects/03179057a85f48688869ec65ba57021a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:11.732006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=03179057a85f48688869ec65ba57021a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:11.734782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:11.764900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:11.771230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caec2af73cc0488a89130a2715503bf3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.771230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caec2af73cc0488a89130a2715503bf3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.771230 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caec2af73cc0488a89130a2715503bf3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.783162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caec2af73cc0488a89130a2715503bf3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.793510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-c1702b10-c0cc-4eff-855b-2f054e4a7af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caec2af73cc0488a89130a2715503bf3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.798107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03179057a85f48688869ec65ba57021a', 'request_id': 'req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.804022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03179057a85f48688869ec65ba57021a', 'request_id': 'req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.813969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03179057a85f48688869ec65ba57021a', 'request_id': 'req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.819439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1024/2061] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:15:11 2022] DELETE /identity/v3/users/caec2af73cc0488a89130a2715503bf3 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:11.823126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a912e6-eeea-4706-88a5-78cded4a95eb None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:11.823423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a912e6-eeea-4706-88a5-78cded4a95eb None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:11.823706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a912e6-eeea-4706-88a5-78cded4a95eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:11.836942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] The token cache is being invalidate because project 03179057a85f48688869ec65ba57021a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:15:11.837237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.837863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d375e5c1-6ac9-4217-a84f-c19d3a7ca65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:11.839459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1038/2062] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:15:11 2022] DELETE /identity/v3/projects/03179057a85f48688869ec65ba57021a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:11.846239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3c9f-c0be-4079-bab7-ba64a12401aa None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:11.846797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3c9f-c0be-4079-bab7-ba64a12401aa None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:11.847437 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf3c9f-c0be-4079-bab7-ba64a12401aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:11.847658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c9a912e6-eeea-4706-88a5-78cded4a95eb None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:11.848884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c9a912e6-eeea-4706-88a5-78cded4a95eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:11.874002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-18bf3c9f-c0be-4079-bab7-ba64a12401aa None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:11.875488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-18bf3c9f-c0be-4079-bab7-ba64a12401aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:11.918187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1025/2063] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:11.928531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a912e6-eeea-4706-88a5-78cded4a95eb None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:11.930195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c9a912e6-eeea-4706-88a5-78cded4a95eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:11.948554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1039/2064] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:11.958473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba67e4-4e88-4170-8580-efe276ec55df None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:11.958781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba67e4-4e88-4170-8580-efe276ec55df None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:11.959076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba67e4-4e88-4170-8580-efe276ec55df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:11.987017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-9eba67e4-4e88-4170-8580-efe276ec55df None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:11.988468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9eba67e4-4e88-4170-8580-efe276ec55df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:11.996542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1040/2065] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:15:12.000442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a912e6-eeea-4706-88a5-78cded4a95eb None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.001077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba67e4-4e88-4170-8580-efe276ec55df None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.004099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c38140-e7a2-4713-8656-0dd2022dc870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.004752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38140-e7a2-4713-8656-0dd2022dc870 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.005036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38140-e7a2-4713-8656-0dd2022dc870 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.005318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c38140-e7a2-4713-8656-0dd2022dc870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.010539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba67e4-4e88-4170-8580-efe276ec55df None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.014809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c38140-e7a2-4713-8656-0dd2022dc870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.015214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.015885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.016168 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.016387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c38140-e7a2-4713-8656-0dd2022dc870 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.016488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.017164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:15:12.017470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.019103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.026390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1026/2066] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:15:12.031487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1041/2067] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:15:12.035973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c38140-e7a2-4713-8656-0dd2022dc870 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.045666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c38140-e7a2-4713-8656-0dd2022dc870 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.050335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d1b9-4aa7-43fb-9ecd-41d9f5d0dfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.051012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d1b9-4aa7-43fb-9ecd-41d9f5d0dfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.051323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d1b9-4aa7-43fb-9ecd-41d9f5d0dfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.051636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8514d1b9-4aa7-43fb-9ecd-41d9f5d0dfe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.052398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514d1b9-4aa7-43fb-9ecd-41d9f5d0dfe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.054387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8514d1b9-4aa7-43fb-9ecd-41d9f5d0dfe7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.084702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1027/2068] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:15:12 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:15:12.086114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.096341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892cf3a0-4dd1-479b-99c4-2c100ed41583 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.104492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:12.162399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe96d58-e651-4d37-97f9-1d8a3802c504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.163187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe96d58-e651-4d37-97f9-1d8a3802c504 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.163495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe96d58-e651-4d37-97f9-1d8a3802c504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.163767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe96d58-e651-4d37-97f9-1d8a3802c504 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.164464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe96d58-e651-4d37-97f9-1d8a3802c504 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.166443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe96d58-e651-4d37-97f9-1d8a3802c504 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.199309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1042/2069] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:15:12 2022] POST /identity/v3/users => generated 285 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:15:12.201938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d1b9-4aa7-43fb-9ecd-41d9f5d0dfe7 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.215721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8514d1b9-4aa7-43fb-9ecd-41d9f5d0dfe7 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.221536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee3454-4f33-44b2-95a9-e0d21f99c897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.222321 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3454-4f33-44b2-95a9-e0d21f99c897 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.222926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3454-4f33-44b2-95a9-e0d21f99c897 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.223244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee3454-4f33-44b2-95a9-e0d21f99c897 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.224170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee3454-4f33-44b2-95a9-e0d21f99c897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.226134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee3454-4f33-44b2-95a9-e0d21f99c897 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.237269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1028/2070] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:15:12.241162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe96d58-e651-4d37-97f9-1d8a3802c504 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.252108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe96d58-e651-4d37-97f9-1d8a3802c504 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.256829 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.257548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.257832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.258100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.258833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.260453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.270678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1043/2071] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:15:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:15:12.275438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee3454-4f33-44b2-95a9-e0d21f99c897 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.286441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee3454-4f33-44b2-95a9-e0d21f99c897 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.291186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb70c92-bec0-4b4e-8d16-7ea27e10d348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.292116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70c92-bec0-4b4e-8d16-7ea27e10d348 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.292400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70c92-bec0-4b4e-8d16-7ea27e10d348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.292685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb70c92-bec0-4b4e-8d16-7ea27e10d348 admin admin] PATH_INFO: `/v3/projects/bbfb0518f0604a5191f8721b1cb692c4/users/ff11bb4f17574ef397c1d7060eabf628/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.308867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.310431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:12.315148 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb70c92-bec0-4b4e-8d16-7ea27e10d348 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbfb0518f0604a5191f8721b1cb692c4, user_id=ff11bb4f17574ef397c1d7060eabf628, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.318380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb70c92-bec0-4b4e-8d16-7ea27e10d348 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.325123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c230cb-a963-4344-b130-5a19e3ae5d97 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.330014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.330877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.331404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.331783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] PATH_INFO: `/v3/projects/d9399282a8ba4f87abe2834702a061b1` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.333341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9399282a8ba4f87abe2834702a061b1)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.335448 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.340594 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1029/2072] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:15:12 2022] PUT /identity/v3/projects/bbfb0518f0604a5191f8721b1cb692c4/users/ff11bb4f17574ef397c1d7060eabf628/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:12.351468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddc606-736e-40d4-81b0-04966a4cdb41 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.351843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddc606-736e-40d4-81b0-04966a4cdb41 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.352196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ddc606-736e-40d4-81b0-04966a4cdb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.375241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:12.386207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-86ddc606-736e-40d4-81b0-04966a4cdb41 None None] MFA Rules not processed for user `ff11bb4f17574ef397c1d7060eabf628`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:12.388276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-86ddc606-736e-40d4-81b0-04966a4cdb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:12.398973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9399282a8ba4f87abe2834702a061b1', 'request_id': 'req-59d0bf45-3a0a-47c3-b12d-5e02310ca034'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.406026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9399282a8ba4f87abe2834702a061b1', 'request_id': 'req-59d0bf45-3a0a-47c3-b12d-5e02310ca034'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.413171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9399282a8ba4f87abe2834702a061b1', 'request_id': 'req-59d0bf45-3a0a-47c3-b12d-5e02310ca034'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.437593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] The token cache is being invalidate because project d9399282a8ba4f87abe2834702a061b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:15:12.437856 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.438461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.440436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1044/2073] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:15:12 2022] DELETE /identity/v3/projects/d9399282a8ba4f87abe2834702a061b1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:12.444218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.445659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:12.462197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1030/2074] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:15:12.477426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddc606-736e-40d4-81b0-04966a4cdb41 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:12.479025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-86ddc606-736e-40d4-81b0-04966a4cdb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:12.516064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0bf45-3a0a-47c3-b12d-5e02310ca034 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.520759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.521634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.521918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.522191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] PATH_INFO: `/v3/users/f16a1a18b66445829ba4b3efb3ad5153` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.523764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16a1a18b66445829ba4b3efb3ad5153)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.526066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.534219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ddc606-736e-40d4-81b0-04966a4cdb41 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:12.538057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d25a45f9ee4ae58c6751a8781d12d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:12.538702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:12.539472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:12.545708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:12.547357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:12.559436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:12.560875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:12.572619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1031/2075] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:15:12.608564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16a1a18b66445829ba4b3efb3ad5153'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.619019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16a1a18b66445829ba4b3efb3ad5153'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.620412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16a1a18b66445829ba4b3efb3ad5153'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.631011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16a1a18b66445829ba4b3efb3ad5153'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.631609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16a1a18b66445829ba4b3efb3ad5153'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:12.659682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1045/2076] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:15:12 2022] DELETE /identity/v3/users/f16a1a18b66445829ba4b3efb3ad5153 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:13.011862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.013444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:13.029493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e281f5-201e-4a13-84bf-f974a4fdf8fc service glance] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.074652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.076386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:13.109568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.112028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.112842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.113197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] PATH_INFO: `/v3/users/f2de40a4164f4460a55c8c5b13c34328` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.114856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2de40a4164f4460a55c8c5b13c34328)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.117293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.163527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a3ca0-8dab-43f1-8d56-b9530a5f8fe8 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.169796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.171023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.171451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.171735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] PATH_INFO: `/v3/projects/218ee4bf994b4493840ab19aca28648a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.173206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=218ee4bf994b4493840ab19aca28648a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.175500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.184350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:13.187715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2de40a4164f4460a55c8c5b13c34328'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.188663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2de40a4164f4460a55c8c5b13c34328'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.190079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2de40a4164f4460a55c8c5b13c34328'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.205405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2de40a4164f4460a55c8c5b13c34328'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.216983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-96642530-85ed-4d67-b14c-0a406b1dc186 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2de40a4164f4460a55c8c5b13c34328'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.239966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1032/2077] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:15:13 2022] DELETE /identity/v3/users/f2de40a4164f4460a55c8c5b13c34328 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:13.244347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aca99a-4826-42cd-99b3-dc12eb5c40e8 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.244635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aca99a-4826-42cd-99b3-dc12eb5c40e8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.244905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aca99a-4826-42cd-99b3-dc12eb5c40e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.250520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '218ee4bf994b4493840ab19aca28648a', 'request_id': 'req-7d637b92-8005-4b54-8fa8-36c0750cdae1'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.260053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '218ee4bf994b4493840ab19aca28648a', 'request_id': 'req-7d637b92-8005-4b54-8fa8-36c0750cdae1'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.267137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '218ee4bf994b4493840ab19aca28648a', 'request_id': 'req-7d637b92-8005-4b54-8fa8-36c0750cdae1'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.271438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-69aca99a-4826-42cd-99b3-dc12eb5c40e8 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:13.272877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-69aca99a-4826-42cd-99b3-dc12eb5c40e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:13.284633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] The token cache is being invalidate because project 218ee4bf994b4493840ab19aca28648a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:15:13.285267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.285935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7d637b92-8005-4b54-8fa8-36c0750cdae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.287919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1046/2078] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:15:13 2022] DELETE /identity/v3/projects/218ee4bf994b4493840ab19aca28648a => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:13.292821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cedb34-6120-4e6f-b90d-70369d7da990 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.293258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cedb34-6120-4e6f-b90d-70369d7da990 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.293548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cedb34-6120-4e6f-b90d-70369d7da990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.313534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-63cedb34-6120-4e6f-b90d-70369d7da990 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:13.314767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-63cedb34-6120-4e6f-b90d-70369d7da990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:13.332963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:13.348644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1033/2079] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:15:13.358118 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aca99a-4826-42cd-99b3-dc12eb5c40e8 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.359917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-69aca99a-4826-42cd-99b3-dc12eb5c40e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:13.383855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1047/2080] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:15:13.385440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c071-54bd-42f9-9062-c883e157049f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.386014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c071-54bd-42f9-9062-c883e157049f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.386296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c071-54bd-42f9-9062-c883e157049f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.404435 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d855c071-54bd-42f9-9062-c883e157049f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:13.405595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d855c071-54bd-42f9-9062-c883e157049f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:13.413699 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1048/2081] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:15:13.417820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855c071-54bd-42f9-9062-c883e157049f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.420795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aca99a-4826-42cd-99b3-dc12eb5c40e8 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.424584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f87341-06cd-4f77-87ff-c4541e289201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.425269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f87341-06cd-4f77-87ff-c4541e289201 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.425549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f87341-06cd-4f77-87ff-c4541e289201 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.425823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f87341-06cd-4f77-87ff-c4541e289201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.426876 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855c071-54bd-42f9-9062-c883e157049f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.431281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.431935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.432210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.432480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.433198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:15:13.433498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.434917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f87341-06cd-4f77-87ff-c4541e289201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.435077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.436402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f87341-06cd-4f77-87ff-c4541e289201 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.445829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1034/2082] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:15:13.446733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1049/2083] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:15:13.450886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f87341-06cd-4f77-87ff-c4541e289201 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.460773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f87341-06cd-4f77-87ff-c4541e289201 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.465819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b93623-7397-4784-9a2c-225125170ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.466590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b93623-7397-4784-9a2c-225125170ca1 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.466875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b93623-7397-4784-9a2c-225125170ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.467232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b93623-7397-4784-9a2c-225125170ca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.468038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b93623-7397-4784-9a2c-225125170ca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.470029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b93623-7397-4784-9a2c-225125170ca1 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.519048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1035/2084] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:15:13 2022] POST /identity/v3/projects => generated 409 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:15:13.523578 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.537582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600edd4c-7376-499d-81e7-a9834dbc1611 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.602720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087378e5-1eb3-4d8e-94aa-d2683f4d14b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.603588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087378e5-1eb3-4d8e-94aa-d2683f4d14b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.603906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087378e5-1eb3-4d8e-94aa-d2683f4d14b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.604151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087378e5-1eb3-4d8e-94aa-d2683f4d14b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.604854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087378e5-1eb3-4d8e-94aa-d2683f4d14b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.607657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087378e5-1eb3-4d8e-94aa-d2683f4d14b1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.637805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1050/2085] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:15:13 2022] POST /identity/v3/users => generated 305 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:15:13.642487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b93623-7397-4784-9a2c-225125170ca1 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.654084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b93623-7397-4784-9a2c-225125170ca1 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.661712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.663605 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.663948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.664213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.665138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.667037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.680665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1036/2086] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:15:13.685173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087378e5-1eb3-4d8e-94aa-d2683f4d14b1 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.699689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087378e5-1eb3-4d8e-94aa-d2683f4d14b1 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.705615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9bcfc-d384-44e4-8999-e0dfea2603b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.706596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bcfc-d384-44e4-8999-e0dfea2603b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.706785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bcfc-d384-44e4-8999-e0dfea2603b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.707088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db9bcfc-d384-44e4-8999-e0dfea2603b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.713356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9bcfc-d384-44e4-8999-e0dfea2603b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.715317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db9bcfc-d384-44e4-8999-e0dfea2603b7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.721949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.723817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:13.737512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1051/2087] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:15:13.741010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9bcfc-d384-44e4-8999-e0dfea2603b7 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.751168 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db9bcfc-d384-44e4-8999-e0dfea2603b7 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.752676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549cd9b1-d709-445c-acc5-30af29c2e159 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:13.757973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba9aa06-f112-498b-b7ad-3507738b0f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.758825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9aa06-f112-498b-b7ad-3507738b0f24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.759109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9aa06-f112-498b-b7ad-3507738b0f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.759596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba9aa06-f112-498b-b7ad-3507738b0f24 admin admin] PATH_INFO: `/v3/projects/946993153a0e449fb1a8bf6caca8de04/users/63a3a8e8f05546ae81abf276bd57163d/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.766022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:13.768672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.768957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.769282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] PATH_INFO: `/v3/projects/98d3bea8a0a8490b95320500190332a2` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.771306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d3bea8a0a8490b95320500190332a2)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.773853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.781949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9aa06-f112-498b-b7ad-3507738b0f24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=946993153a0e449fb1a8bf6caca8de04, user_id=63a3a8e8f05546ae81abf276bd57163d, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:13.784752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba9aa06-f112-498b-b7ad-3507738b0f24 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:13.804756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1052/2088] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:15:13 2022] PUT /identity/v3/projects/946993153a0e449fb1a8bf6caca8de04/users/63a3a8e8f05546ae81abf276bd57163d/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:13.810779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:13.816646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782f3ea-22fa-494b-b8e3-0dc2d786cc4f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:13.816646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782f3ea-22fa-494b-b8e3-0dc2d786cc4f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:13.816879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782f3ea-22fa-494b-b8e3-0dc2d786cc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:13.829782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d3bea8a0a8490b95320500190332a2', 'request_id': 'req-2a309eeb-fc2a-4e3b-960a-84a468c1568f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.838661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d3bea8a0a8490b95320500190332a2', 'request_id': 'req-2a309eeb-fc2a-4e3b-960a-84a468c1568f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.843478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a782f3ea-22fa-494b-b8e3-0dc2d786cc4f None None] MFA Rules not processed for user `63a3a8e8f05546ae81abf276bd57163d`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:13.844683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a782f3ea-22fa-494b-b8e3-0dc2d786cc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:13.844837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d3bea8a0a8490b95320500190332a2', 'request_id': 'req-2a309eeb-fc2a-4e3b-960a-84a468c1568f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.857547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] The token cache is being invalidate because project 98d3bea8a0a8490b95320500190332a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:15:13.857950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.858585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:13.860556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1037/2089] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:15:13 2022] DELETE /identity/v3/projects/98d3bea8a0a8490b95320500190332a2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:15:13.904816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:13.927394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1053/2090] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:13.932302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:13.935651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:14.027296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a309eeb-fc2a-4e3b-960a-84a468c1568f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:14.036904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4074c0-96e3-499a-afd7-e4b0fb73c2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:14.037607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4074c0-96e3-499a-afd7-e4b0fb73c2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.037886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4074c0-96e3-499a-afd7-e4b0fb73c2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.038158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4074c0-96e3-499a-afd7-e4b0fb73c2c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.038938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4074c0-96e3-499a-afd7-e4b0fb73c2c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:14.040944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4074c0-96e3-499a-afd7-e4b0fb73c2c4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:14.074546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1038/2091] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:15:13 2022] POST /identity/v3/projects => generated 409 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:15:14.078767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782f3ea-22fa-494b-b8e3-0dc2d786cc4f None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:14.089458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782f3ea-22fa-494b-b8e3-0dc2d786cc4f None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:14.155144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add83586-c43d-4d5e-ba57-003f77c04754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:14.156156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83586-c43d-4d5e-ba57-003f77c04754 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.156318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83586-c43d-4d5e-ba57-003f77c04754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.156832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83586-c43d-4d5e-ba57-003f77c04754 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.157997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add83586-c43d-4d5e-ba57-003f77c04754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:14.160057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add83586-c43d-4d5e-ba57-003f77c04754 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:14.180718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1054/2092] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:15:14 2022] POST /identity/v3/users => generated 311 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:15:14.184650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4074c0-96e3-499a-afd7-e4b0fb73c2c4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:14.196496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4074c0-96e3-499a-afd7-e4b0fb73c2c4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:14.201236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6f69c-502f-4cc8-ad5f-6e2c9f795613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:14.201906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6f69c-502f-4cc8-ad5f-6e2c9f795613 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.202183 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6f69c-502f-4cc8-ad5f-6e2c9f795613 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.202455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe6f69c-502f-4cc8-ad5f-6e2c9f795613 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.203191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6f69c-502f-4cc8-ad5f-6e2c9f795613 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:14.204833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe6f69c-502f-4cc8-ad5f-6e2c9f795613 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:14.215651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1039/2093] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:15:14.220008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add83586-c43d-4d5e-ba57-003f77c04754 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:14.231598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add83586-c43d-4d5e-ba57-003f77c04754 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:14.237775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0b1cd-f207-4437-99e6-3b38d0b03c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:14.237775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0b1cd-f207-4437-99e6-3b38d0b03c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.237775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0b1cd-f207-4437-99e6-3b38d0b03c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.239197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c0b1cd-f207-4437-99e6-3b38d0b03c3d admin admin] PATH_INFO: `/v3/projects/0e5fad626987478a85aa88a12182829e/users/bdda9566eae647538bedbca034f03503/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.255291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0b1cd-f207-4437-99e6-3b38d0b03c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5fad626987478a85aa88a12182829e, user_id=bdda9566eae647538bedbca034f03503, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:14.258180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c0b1cd-f207-4437-99e6-3b38d0b03c3d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:14.275414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1055/2094] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:15:14 2022] PUT /identity/v3/projects/0e5fad626987478a85aa88a12182829e/users/bdda9566eae647538bedbca034f03503/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:15:14.280572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6f69c-502f-4cc8-ad5f-6e2c9f795613 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:14.290882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe6f69c-502f-4cc8-ad5f-6e2c9f795613 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:14.352930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7331c0e-d102-4a18-bd5a-2c51d080578a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:14.353712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7331c0e-d102-4a18-bd5a-2c51d080578a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.354081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7331c0e-d102-4a18-bd5a-2c51d080578a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.354380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7331c0e-d102-4a18-bd5a-2c51d080578a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.355250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7331c0e-d102-4a18-bd5a-2c51d080578a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:14.356958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7331c0e-d102-4a18-bd5a-2c51d080578a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:14.367590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1040/2095] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:15:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:15:14.373751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0b1cd-f207-4437-99e6-3b38d0b03c3d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:14.384512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c0b1cd-f207-4437-99e6-3b38d0b03c3d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:14.389595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:14.390477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.390753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.391036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] PATH_INFO: `/v3/projects/0e5fad626987478a85aa88a12182829e/users/bdda9566eae647538bedbca034f03503/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.393121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e5fad626987478a85aa88a12182829e, user_id=bdda9566eae647538bedbca034f03503, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:14.396519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:14.407755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1056/2096] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:15:14 2022] PUT /identity/v3/projects/0e5fad626987478a85aa88a12182829e/users/bdda9566eae647538bedbca034f03503/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:15:14.419222 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e5f5f-0132-41ef-bbf6-89c66fcca7ca None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.419539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e5f5f-0132-41ef-bbf6-89c66fcca7ca None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.419817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e5f5f-0132-41ef-bbf6-89c66fcca7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.449920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f76e5f5f-0132-41ef-bbf6-89c66fcca7ca None None] MFA Rules not processed for user `bdda9566eae647538bedbca034f03503`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:14.451075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f76e5f5f-0132-41ef-bbf6-89c66fcca7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:14.464158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:14.530002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1041/2097] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:14 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:14.540896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:14.542665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:14.603393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0533b6c-8caf-45e9-9861-cafb27d945ec admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:14.608006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95a91f-2f2c-49fb-8017-c6f4a66ae7fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:14.608667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a91f-2f2c-49fb-8017-c6f4a66ae7fa service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.608946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a91f-2f2c-49fb-8017-c6f4a66ae7fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.609219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95a91f-2f2c-49fb-8017-c6f4a66ae7fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.620022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95a91f-2f2c-49fb-8017-c6f4a66ae7fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:14.621473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95a91f-2f2c-49fb-8017-c6f4a66ae7fa service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:14.631390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1057/2098] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:14 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:15:14.809714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e5f5f-0132-41ef-bbf6-89c66fcca7ca None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:14.820692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e5f5f-0132-41ef-bbf6-89c66fcca7ca None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:14.828417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc40a9-4b65-4f71-ae8f-0e6c8136513c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:14.829110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc40a9-4b65-4f71-ae8f-0e6c8136513c service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:14.829455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc40a9-4b65-4f71-ae8f-0e6c8136513c service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:14.829733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc40a9-4b65-4f71-ae8f-0e6c8136513c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:14.893654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc40a9-4b65-4f71-ae8f-0e6c8136513c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:14.895089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc40a9-4b65-4f71-ae8f-0e6c8136513c service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:14.904397 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1042/2099] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:14 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:15:23.354816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95a91f-2f2c-49fb-8017-c6f4a66ae7fa service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:23.400305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:23.466572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95a91f-2f2c-49fb-8017-c6f4a66ae7fa service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:23.471587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:23.472460 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:23.472777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:23.473049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] PATH_INFO: `/v3/users/63a3a8e8f05546ae81abf276bd57163d` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:23.474284 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=63a3a8e8f05546ae81abf276bd57163d)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:23.476252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:23.536534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63a3a8e8f05546ae81abf276bd57163d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:23.546928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63a3a8e8f05546ae81abf276bd57163d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:23.547791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63a3a8e8f05546ae81abf276bd57163d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:23.558154 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63a3a8e8f05546ae81abf276bd57163d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:23.558833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63a3a8e8f05546ae81abf276bd57163d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:23.586390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1058/2100] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:15:23 2022] DELETE /identity/v3/users/63a3a8e8f05546ae81abf276bd57163d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:23.591572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae948-5b1c-4843-be7d-c0a1bb46a816 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:23.592075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae948-5b1c-4843-be7d-c0a1bb46a816 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:23.592187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ae948-5b1c-4843-be7d-c0a1bb46a816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:23.655033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-cc2ae948-5b1c-4843-be7d-c0a1bb46a816 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:23.656289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-cc2ae948-5b1c-4843-be7d-c0a1bb46a816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:23.719773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1043/2101] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:15:23.726579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:23.727986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:23.851137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:23.865363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb6f19-d291-4b72-a9ed-2c925b0d02ac admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:23.869634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd9181-9902-4ff8-aa4d-8f8812e43408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:23.870270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd9181-9902-4ff8-aa4d-8f8812e43408 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:23.870550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd9181-9902-4ff8-aa4d-8f8812e43408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:23.870947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd9181-9902-4ff8-aa4d-8f8812e43408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:23.895134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd9181-9902-4ff8-aa4d-8f8812e43408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:23.899784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd9181-9902-4ff8-aa4d-8f8812e43408 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:23.924743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1059/2102] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:15:24.271753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ae948-5b1c-4843-be7d-c0a1bb46a816 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:24.273787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-cc2ae948-5b1c-4843-be7d-c0a1bb46a816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:24.297523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ae948-5b1c-4843-be7d-c0a1bb46a816 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:24.307589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:24.308900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:24.309250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:24.309627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] PATH_INFO: `/v3/projects/946993153a0e449fb1a8bf6caca8de04` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:24.311074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] RBAC: Authorizing `identity:delete_project(project_id=946993153a0e449fb1a8bf6caca8de04)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:24.313385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:24.337347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:24.376337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '946993153a0e449fb1a8bf6caca8de04', 'request_id': 'req-e08100ab-eb76-49de-b083-31f579d6327b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.386430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '946993153a0e449fb1a8bf6caca8de04', 'request_id': 'req-e08100ab-eb76-49de-b083-31f579d6327b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.398132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '946993153a0e449fb1a8bf6caca8de04', 'request_id': 'req-e08100ab-eb76-49de-b083-31f579d6327b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.413280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] The token cache is being invalidate because project 946993153a0e449fb1a8bf6caca8de04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:15:24.413768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.414500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.416348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1044/2103] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:15:24 2022] DELETE /identity/v3/projects/946993153a0e449fb1a8bf6caca8de04 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:24.420796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd9181-9902-4ff8-aa4d-8f8812e43408 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:24.432938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd9181-9902-4ff8-aa4d-8f8812e43408 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:24.492285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:24.493072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:24.493342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:24.493635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] PATH_INFO: `/v3/users/bdda9566eae647538bedbca034f03503` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:24.494893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdda9566eae647538bedbca034f03503)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:24.496869 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:24.543968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdda9566eae647538bedbca034f03503'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.551284 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdda9566eae647538bedbca034f03503'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.551981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdda9566eae647538bedbca034f03503'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.562297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdda9566eae647538bedbca034f03503'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.563340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ae765d99-eebd-4eee-8a43-1aec2cb9b92f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdda9566eae647538bedbca034f03503'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:24.585371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1060/2104] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:15:24 2022] DELETE /identity/v3/users/bdda9566eae647538bedbca034f03503 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:24.862297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:24.864009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:24.881572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08100ab-eb76-49de-b083-31f579d6327b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:24.948609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:24.949485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:24.949764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:24.950039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] PATH_INFO: `/v3/projects/0e5fad626987478a85aa88a12182829e` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:24.951286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e5fad626987478a85aa88a12182829e)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:24.953352 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:25.006250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e5fad626987478a85aa88a12182829e', 'request_id': 'req-0738678c-b2eb-4379-8466-6291768e9137'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:25.015816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e5fad626987478a85aa88a12182829e', 'request_id': 'req-0738678c-b2eb-4379-8466-6291768e9137'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:25.021376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e5fad626987478a85aa88a12182829e', 'request_id': 'req-0738678c-b2eb-4379-8466-6291768e9137'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:25.033450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] The token cache is being invalidate because project 0e5fad626987478a85aa88a12182829e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:15:25.033670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:25.034478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0738678c-b2eb-4379-8466-6291768e9137 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:25.036215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1045/2105] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:15:24 2022] DELETE /identity/v3/projects/0e5fad626987478a85aa88a12182829e => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:25.041449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3af76-b13d-4519-b50c-af63110b7e2d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.041813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3af76-b13d-4519-b50c-af63110b7e2d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.042102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3af76-b13d-4519-b50c-af63110b7e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.083059 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2ef3af76-b13d-4519-b50c-af63110b7e2d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:25.084383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2ef3af76-b13d-4519-b50c-af63110b7e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:25.141556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:25.162132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1061/2106] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:25.170236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dc0cd-a071-4943-ba4a-bae8e2ae8ae9 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.170525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dc0cd-a071-4943-ba4a-bae8e2ae8ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.170805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053dc0cd-a071-4943-ba4a-bae8e2ae8ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.189697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-053dc0cd-a071-4943-ba4a-bae8e2ae8ae9 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:25.190925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-053dc0cd-a071-4943-ba4a-bae8e2ae8ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:25.199075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1046/2107] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:25.203447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3af76-b13d-4519-b50c-af63110b7e2d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:25.213572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3af76-b13d-4519-b50c-af63110b7e2d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:25.218661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:25.219324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.219626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.219897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.220632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:15:25.220937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:25.222683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:25.235651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1062/2108] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:15:25.241316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dc0cd-a071-4943-ba4a-bae8e2ae8ae9 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:25.250686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053dc0cd-a071-4943-ba4a-bae8e2ae8ae9 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:25.255429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0a5b-3c91-4512-aff1-dbcc9b14078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:25.256110 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0a5b-3c91-4512-aff1-dbcc9b14078b admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.256392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0a5b-3c91-4512-aff1-dbcc9b14078b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.256667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ae0a5b-3c91-4512-aff1-dbcc9b14078b admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.257364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae0a5b-3c91-4512-aff1-dbcc9b14078b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:25.259311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ae0a5b-3c91-4512-aff1-dbcc9b14078b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:25.288471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1047/2109] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:15:25 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:15:25.292900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:25.303365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d042900-4039-42e9-9afd-ddee2b11c3a7 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:25.363986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4fc3a-32f2-463e-91c9-6251279e1327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:25.364668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4fc3a-32f2-463e-91c9-6251279e1327 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.364952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4fc3a-32f2-463e-91c9-6251279e1327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.365236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf4fc3a-32f2-463e-91c9-6251279e1327 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.366012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4fc3a-32f2-463e-91c9-6251279e1327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:25.368044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf4fc3a-32f2-463e-91c9-6251279e1327 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:25.393993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1063/2110] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:15:25 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:15:25.395347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0a5b-3c91-4512-aff1-dbcc9b14078b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:25.406415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ae0a5b-3c91-4512-aff1-dbcc9b14078b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:25.411693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98364e8-5eb4-401f-b708-48b2d57e680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:25.412418 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98364e8-5eb4-401f-b708-48b2d57e680e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.412797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98364e8-5eb4-401f-b708-48b2d57e680e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.412993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98364e8-5eb4-401f-b708-48b2d57e680e admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.413785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98364e8-5eb4-401f-b708-48b2d57e680e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:25.415454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98364e8-5eb4-401f-b708-48b2d57e680e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:25.416947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:25.429354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1048/2111] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:15:25.432458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4fc3a-32f2-463e-91c9-6251279e1327 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:25.442521 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf4fc3a-32f2-463e-91c9-6251279e1327 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:25.447282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f444827c-3a14-4681-ac3c-00dfe584cdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:25.447996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444827c-3a14-4681-ac3c-00dfe584cdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.448280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444827c-3a14-4681-ac3c-00dfe584cdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.448555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f444827c-3a14-4681-ac3c-00dfe584cdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.449300 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444827c-3a14-4681-ac3c-00dfe584cdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:25.450932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f444827c-3a14-4681-ac3c-00dfe584cdb9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:25.461279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1064/2112] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:15:25.469277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98364e8-5eb4-401f-b708-48b2d57e680e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:25.479843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98364e8-5eb4-401f-b708-48b2d57e680e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:25.485286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:25.486202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.486479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.486769 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] PATH_INFO: `/v3/projects/db434a374b2a48bb9e96885d88a2c7ee/users/29381071d26a4fbcb53efb53408297a1/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.502022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db434a374b2a48bb9e96885d88a2c7ee, user_id=29381071d26a4fbcb53efb53408297a1, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:25.504710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:25.521745 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1049/2113] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:15:25 2022] PUT /identity/v3/projects/db434a374b2a48bb9e96885d88a2c7ee/users/29381071d26a4fbcb53efb53408297a1/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:25.532604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686cdb-0645-4a1d-826d-a1f1e5e4d2d6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.532892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686cdb-0645-4a1d-826d-a1f1e5e4d2d6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.533162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84686cdb-0645-4a1d-826d-a1f1e5e4d2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.562107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-84686cdb-0645-4a1d-826d-a1f1e5e4d2d6 None None] MFA Rules not processed for user `29381071d26a4fbcb53efb53408297a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:25.563245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-84686cdb-0645-4a1d-826d-a1f1e5e4d2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:25.631547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1065/2114] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:25.643073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:25.644917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:25.704851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76752b2-678e-44fd-874f-73be86ad84a3 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:25.709252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d25a45f9ee4ae58c6751a8781d12d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:25.709958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:25.710241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:25.710657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:25.719515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:25.720984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:25.730171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1050/2115] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:15:32.953958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30385-4062-41f9-8274-ea7d1303cb4e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:32.954474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30385-4062-41f9-8274-ea7d1303cb4e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:32.954564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da30385-4062-41f9-8274-ea7d1303cb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:32.990025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1da30385-4062-41f9-8274-ea7d1303cb4e None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:32.991354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1da30385-4062-41f9-8274-ea7d1303cb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:33.028994 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:33.053791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1066/2116] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:15:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:33.067475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:33.068978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:33.142043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3224587f-f004-41ff-9fca-d8bc7933e31b service glance] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:33.147285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd9daca-afc9-4e99-99d4-d3fe3bcd1bf0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:33.148304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9daca-afc9-4e99-99d4-d3fe3bcd1bf0 service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:33.148620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9daca-afc9-4e99-99d4-d3fe3bcd1bf0 service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:33.148864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd9daca-afc9-4e99-99d4-d3fe3bcd1bf0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:33.158278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9daca-afc9-4e99-99d4-d3fe3bcd1bf0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:33.159990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd9daca-afc9-4e99-99d4-d3fe3bcd1bf0 service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:33.168365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1051/2117] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:15:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:15:36.988519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf970d7-9226-4521-9cd8-6a12e407282c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:36.989177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf970d7-9226-4521-9cd8-6a12e407282c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:36.989332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf970d7-9226-4521-9cd8-6a12e407282c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:37.010794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4bf970d7-9226-4521-9cd8-6a12e407282c None None] MFA Rules not processed for user `ff944e4cf29b4297acc63b591d90daaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:37.012037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4bf970d7-9226-4521-9cd8-6a12e407282c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:37.075179 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1067/2118] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:43.165078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=70,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:47.068273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=33 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:56.472870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812a91d-50e7-4403-b351-f777d32db145 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:56.473362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812a91d-50e7-4403-b351-f777d32db145 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:56.473447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812a91d-50e7-4403-b351-f777d32db145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:56.492528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-2812a91d-50e7-4403-b351-f777d32db145 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:56.493800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2812a91d-50e7-4403-b351-f777d32db145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:56.507603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1052/2119] 10.222.0.69 () {58 vars in 967 bytes} [Fri May 20 09:15:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:56.520218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf970d7-9226-4521-9cd8-6a12e407282c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:56.533782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf970d7-9226-4521-9cd8-6a12e407282c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:56.539022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc95a5-238a-4ed2-a19b-47d60dc6d7c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:56.540125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc95a5-238a-4ed2-a19b-47d60dc6d7c8 service swift] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:56.540406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc95a5-238a-4ed2-a19b-47d60dc6d7c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:56.540727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc95a5-238a-4ed2-a19b-47d60dc6d7c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:56.554665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc95a5-238a-4ed2-a19b-47d60dc6d7c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:56.557254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc95a5-238a-4ed2-a19b-47d60dc6d7c8 service swift] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:56.566741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1068/2120] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:15:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:15:58.634666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812a91d-50e7-4403-b351-f777d32db145 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:58.699704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812a91d-50e7-4403-b351-f777d32db145 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:58.704528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:58.705408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:58.705687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:58.705961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] PATH_INFO: `/v3/users/29381071d26a4fbcb53efb53408297a1` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:58.707175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29381071d26a4fbcb53efb53408297a1)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:58.709147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:58.756906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29381071d26a4fbcb53efb53408297a1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:58.757727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29381071d26a4fbcb53efb53408297a1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:58.758282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29381071d26a4fbcb53efb53408297a1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:58.765501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29381071d26a4fbcb53efb53408297a1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:58.776653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29381071d26a4fbcb53efb53408297a1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:58.802701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf82b4-5011-4092-b4f5-3a86def2a527 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:58.803211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1053/2121] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:15:58 2022] DELETE /identity/v3/users/29381071d26a4fbcb53efb53408297a1 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:58.803672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf82b4-5011-4092-b4f5-3a86def2a527 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:58.803957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf82b4-5011-4092-b4f5-3a86def2a527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:58.837391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-75bf82b4-5011-4092-b4f5-3a86def2a527 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:58.838579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-75bf82b4-5011-4092-b4f5-3a86def2a527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:58.939287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1069/2122] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:15:58.950519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:58.951451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:59.021615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b470d-7998-49da-a060-4ce372ae6183 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:59.028013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e5710-7e41-47de-a5b4-9b97245de731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:59.028775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e5710-7e41-47de-a5b4-9b97245de731 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:59.029123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e5710-7e41-47de-a5b4-9b97245de731 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:59.029461 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e5710-7e41-47de-a5b4-9b97245de731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:59.039450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e5710-7e41-47de-a5b4-9b97245de731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:59.040933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e5710-7e41-47de-a5b4-9b97245de731 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:59.051863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1054/2123] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:15:59.409664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf82b4-5011-4092-b4f5-3a86def2a527 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:59.411279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-75bf82b4-5011-4092-b4f5-3a86def2a527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:59.431183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf82b4-5011-4092-b4f5-3a86def2a527 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:59.437169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:59.438443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:59.438873 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:59.439321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] PATH_INFO: `/v3/projects/db434a374b2a48bb9e96885d88a2c7ee` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:59.440889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] RBAC: Authorizing `identity:delete_project(project_id=db434a374b2a48bb9e96885d88a2c7ee)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:59.443801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:59.563196 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db434a374b2a48bb9e96885d88a2c7ee', 'request_id': 'req-0996ddac-1f13-4c41-9a20-15504d98777f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:59.570396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db434a374b2a48bb9e96885d88a2c7ee', 'request_id': 'req-0996ddac-1f13-4c41-9a20-15504d98777f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:59.585198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db434a374b2a48bb9e96885d88a2c7ee', 'request_id': 'req-0996ddac-1f13-4c41-9a20-15504d98777f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:59.608467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] The token cache is being invalidate because project db434a374b2a48bb9e96885d88a2c7ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:15:59.609024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:59.609950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-0996ddac-1f13-4c41-9a20-15504d98777f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:15:59.612334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1070/2124] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:15:59 2022] DELETE /identity/v3/projects/db434a374b2a48bb9e96885d88a2c7ee => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:15:59.617276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe8b1c-6cff-4277-95df-e74171dd2f46 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:59.617565 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe8b1c-6cff-4277-95df-e74171dd2f46 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:59.618417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe8b1c-6cff-4277-95df-e74171dd2f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:59.629528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:15:59.655334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-babe8b1c-6cff-4277-95df-e74171dd2f46 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:59.656616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-babe8b1c-6cff-4277-95df-e74171dd2f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:59.737369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1055/2125] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:59.745204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca5eb-e76a-4689-9618-27a5d5b94e72 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:59.745524 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca5eb-e76a-4689-9618-27a5d5b94e72 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:59.745803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ca5eb-e76a-4689-9618-27a5d5b94e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:59.767846 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-fe9ca5eb-e76a-4689-9618-27a5d5b94e72 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:15:59.771362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-fe9ca5eb-e76a-4689-9618-27a5d5b94e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:15:59.786557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1071/2126] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:15:59.791172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe8b1c-6cff-4277-95df-e74171dd2f46 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:59.805130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe8b1c-6cff-4277-95df-e74171dd2f46 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:59.810641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:59.811842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:59.812486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:59.812968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:59.815141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:15:59.815728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:59.817581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:59.833283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1056/2127] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:15:59.837477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ca5eb-e76a-4689-9618-27a5d5b94e72 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:59.847878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ca5eb-e76a-4689-9618-27a5d5b94e72 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:59.852682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:15:59.853351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:15:59.853634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:15:59.853908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:15:59.854618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:15:59.857071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:15:59.888614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1072/2128] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:15:59 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:15:59.893271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:59.907017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b3feb-35ef-4be8-a48f-b06ac1c59e5e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:15:59.951610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:15:59.953543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:00.002696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44c77-4d3b-4afe-b2ad-4fe96b29218c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.003502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44c77-4d3b-4afe-b2ad-4fe96b29218c admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.004734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44c77-4d3b-4afe-b2ad-4fe96b29218c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.005250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff44c77-4d3b-4afe-b2ad-4fe96b29218c admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.008036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff44c77-4d3b-4afe-b2ad-4fe96b29218c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.010104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff44c77-4d3b-4afe-b2ad-4fe96b29218c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.044004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1057/2129] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:15:59 2022] POST /identity/v3/users => generated 289 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:00.044224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:00.045463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44c77-4d3b-4afe-b2ad-4fe96b29218c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.057051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff44c77-4d3b-4afe-b2ad-4fe96b29218c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.062335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5381f58-07a4-4fd3-a79b-cb451c00e773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.062945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5381f58-07a4-4fd3-a79b-cb451c00e773 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.063226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5381f58-07a4-4fd3-a79b-cb451c00e773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.063604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5381f58-07a4-4fd3-a79b-cb451c00e773 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.064485 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5381f58-07a4-4fd3-a79b-cb451c00e773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.066232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5381f58-07a4-4fd3-a79b-cb451c00e773 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.070235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c32a14c-7871-4d42-8334-08903c42858c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.076241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.077746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1058/2130] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:00.078309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.078737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.079180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] PATH_INFO: `/v3/users/712133b246c94438b94c1f865dca1be9` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.080802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=712133b246c94438b94c1f865dca1be9)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.081657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5381f58-07a4-4fd3-a79b-cb451c00e773 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.082775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.097412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5381f58-07a4-4fd3-a79b-cb451c00e773 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.104030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2749ed7-bce9-4166-8692-bb0b7f6212d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.104804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2749ed7-bce9-4166-8692-bb0b7f6212d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.105090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2749ed7-bce9-4166-8692-bb0b7f6212d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.105368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2749ed7-bce9-4166-8692-bb0b7f6212d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.106118 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2749ed7-bce9-4166-8692-bb0b7f6212d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.107881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2749ed7-bce9-4166-8692-bb0b7f6212d4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.121938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1059/2131] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:00.126287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2749ed7-bce9-4166-8692-bb0b7f6212d4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.151428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2749ed7-bce9-4166-8692-bb0b7f6212d4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.164662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f423f02-b4eb-4e6d-8f50-4cd81c56ebb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.165592 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f423f02-b4eb-4e6d-8f50-4cd81c56ebb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.165914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f423f02-b4eb-4e6d-8f50-4cd81c56ebb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.166196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f423f02-b4eb-4e6d-8f50-4cd81c56ebb1 admin admin] PATH_INFO: `/v3/projects/9393be3d2e22466f97efd28482358a7b/users/1f1b74fb31e642eca02d095a7292aacd/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.184495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '712133b246c94438b94c1f865dca1be9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.193043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f423f02-b4eb-4e6d-8f50-4cd81c56ebb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9393be3d2e22466f97efd28482358a7b, user_id=1f1b74fb31e642eca02d095a7292aacd, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.197628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f423f02-b4eb-4e6d-8f50-4cd81c56ebb1 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.200116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '712133b246c94438b94c1f865dca1be9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.201998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '712133b246c94438b94c1f865dca1be9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.212597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '712133b246c94438b94c1f865dca1be9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.214336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '712133b246c94438b94c1f865dca1be9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.219217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1060/2132] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:00 2022] PUT /identity/v3/projects/9393be3d2e22466f97efd28482358a7b/users/1f1b74fb31e642eca02d095a7292aacd/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:16:00.230235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24632-2bb5-4c2a-b1db-c8af60cb32df None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.231016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24632-2bb5-4c2a-b1db-c8af60cb32df None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.231369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e24632-2bb5-4c2a-b1db-c8af60cb32df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.240891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1073/2133] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:15:59 2022] DELETE /identity/v3/users/712133b246c94438b94c1f865dca1be9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:16:00.265654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e0e24632-2bb5-4c2a-b1db-c8af60cb32df None None] MFA Rules not processed for user `1f1b74fb31e642eca02d095a7292aacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:00.267012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e0e24632-2bb5-4c2a-b1db-c8af60cb32df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:00.291613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:00.346092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1061/2134] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:16:00.350281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.352339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:00.448762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bff78-0457-4758-9f5a-bed6733f36ba admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.454741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a44a1-e96e-4eb8-b865-a38159ba8fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.455748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a44a1-e96e-4eb8-b865-a38159ba8fdb admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.457712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a44a1-e96e-4eb8-b865-a38159ba8fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.457712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51a44a1-e96e-4eb8-b865-a38159ba8fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.457712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a44a1-e96e-4eb8-b865-a38159ba8fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.459973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51a44a1-e96e-4eb8-b865-a38159ba8fdb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.494505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1074/2135] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:16:00 2022] POST /identity/v3/projects => generated 375 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:00.499340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24632-2bb5-4c2a-b1db-c8af60cb32df None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.511674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e24632-2bb5-4c2a-b1db-c8af60cb32df None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.586639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.589143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.589143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.589143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.592272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.594688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.623707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1062/2136] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:16:00 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:00.627678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a44a1-e96e-4eb8-b865-a38159ba8fdb admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.632091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.634433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:00.639389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51a44a1-e96e-4eb8-b865-a38159ba8fdb admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.644939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871aa1d-8b62-46e9-9958-4429b22d316b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.645809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871aa1d-8b62-46e9-9958-4429b22d316b admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.646248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871aa1d-8b62-46e9-9958-4429b22d316b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.646500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871aa1d-8b62-46e9-9958-4429b22d316b admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.647777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871aa1d-8b62-46e9-9958-4429b22d316b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.649500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f871aa1d-8b62-46e9-9958-4429b22d316b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.649900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f3e64-845a-4a22-b82d-c3efb12b3a38 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.655400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.656993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.657629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.658103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] PATH_INFO: `/v3/projects/7264062888b544cfb7fbc74cc87f386f` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.659536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] RBAC: Authorizing `identity:delete_project(project_id=7264062888b544cfb7fbc74cc87f386f)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.661662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.662659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1075/2137] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:00.666307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871aa1d-8b62-46e9-9958-4429b22d316b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.678649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f871aa1d-8b62-46e9-9958-4429b22d316b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.684516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ed33b-bae7-479c-ac0a-f03d8a375b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.685291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ed33b-bae7-479c-ac0a-f03d8a375b32 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.685641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ed33b-bae7-479c-ac0a-f03d8a375b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.686006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2ed33b-bae7-479c-ac0a-f03d8a375b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.686999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ed33b-bae7-479c-ac0a-f03d8a375b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.688951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2ed33b-bae7-479c-ac0a-f03d8a375b32 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.716508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1076/2138] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:16:00.717658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ed33b-bae7-479c-ac0a-f03d8a375b32 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.723502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7264062888b544cfb7fbc74cc87f386f', 'request_id': 'req-b5d63183-3d0c-4679-9a70-f9fb8bbccace'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.729256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2ed33b-bae7-479c-ac0a-f03d8a375b32 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.736213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7264062888b544cfb7fbc74cc87f386f', 'request_id': 'req-b5d63183-3d0c-4679-9a70-f9fb8bbccace'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.745197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7264062888b544cfb7fbc74cc87f386f', 'request_id': 'req-b5d63183-3d0c-4679-9a70-f9fb8bbccace'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.752333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:00.764365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] The token cache is being invalidate because project 7264062888b544cfb7fbc74cc87f386f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:16:00.765122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.773008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b5d63183-3d0c-4679-9a70-f9fb8bbccace admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:00.774914 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1063/2139] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:16:00 2022] DELETE /identity/v3/projects/7264062888b544cfb7fbc74cc87f386f => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:16:00.780125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296e5c-b50e-4bb1-8af6-edbd67ffa14f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.780406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296e5c-b50e-4bb1-8af6-edbd67ffa14f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.780686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84296e5c-b50e-4bb1-8af6-edbd67ffa14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.802054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-84296e5c-b50e-4bb1-8af6-edbd67ffa14f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:00.803276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-84296e5c-b50e-4bb1-8af6-edbd67ffa14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:00.809552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113a8d6-72f0-4648-8892-8ff48dd5b608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.810636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113a8d6-72f0-4648-8892-8ff48dd5b608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.810982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113a8d6-72f0-4648-8892-8ff48dd5b608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.811369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113a8d6-72f0-4648-8892-8ff48dd5b608 admin admin] PATH_INFO: `/v3/projects/445ed1dff2f14ea0a299bdaf2cde31a7/users/15f43038911f4ddfadf2ded599f038bf/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.814655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1064/2140] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:16:00.824014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6cf58-9bf9-4b72-88f4-919f5b58ca18 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.824709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6cf58-9bf9-4b72-88f4-919f5b58ca18 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.825710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6cf58-9bf9-4b72-88f4-919f5b58ca18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.834061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113a8d6-72f0-4648-8892-8ff48dd5b608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=445ed1dff2f14ea0a299bdaf2cde31a7, user_id=15f43038911f4ddfadf2ded599f038bf, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.839987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113a8d6-72f0-4648-8892-8ff48dd5b608 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:00.853366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4ad6cf58-9bf9-4b72-88f4-919f5b58ca18 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:00.854722 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4ad6cf58-9bf9-4b72-88f4-919f5b58ca18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:00.859673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1077/2141] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:00 2022] PUT /identity/v3/projects/445ed1dff2f14ea0a299bdaf2cde31a7/users/15f43038911f4ddfadf2ded599f038bf/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:16:00.861306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee3d56-3521-4aa1-a547-f31ab0ce4163 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.861852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee3d56-3521-4aa1-a547-f31ab0ce4163 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.862199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ee3d56-3521-4aa1-a547-f31ab0ce4163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.888957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-61ee3d56-3521-4aa1-a547-f31ab0ce4163 None None] MFA Rules not processed for user `992571802b7d426f923e0f0bc41e28de`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:00.890431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-61ee3d56-3521-4aa1-a547-f31ab0ce4163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:00.891970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:00.948065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1065/2142] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:00.952341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0928a-9e27-4b59-bf4c-3d5ce751a741 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.952803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0928a-9e27-4b59-bf4c-3d5ce751a741 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.953084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0928a-9e27-4b59-bf4c-3d5ce751a741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.971927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1078/2143] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:00.973042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee3d56-3521-4aa1-a547-f31ab0ce4163 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:00.982990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c8a0928a-9e27-4b59-bf4c-3d5ce751a741 None None] MFA Rules not processed for user `15f43038911f4ddfadf2ded599f038bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:00.985082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c8a0928a-9e27-4b59-bf4c-3d5ce751a741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:00.987328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ee3d56-3521-4aa1-a547-f31ab0ce4163 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:00.992343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:00.993205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:00.993641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:00.994096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:00.994981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:16:00.995536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:00.997428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.015106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1079/2144] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:16:01.019232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.040111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ed212-eb97-4ef4-9f22-a2c225373b08 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.047610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb40ae-5c37-4cb6-b566-703043171a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.048429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb40ae-5c37-4cb6-b566-703043171a13 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.048716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb40ae-5c37-4cb6-b566-703043171a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.048998 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eb40ae-5c37-4cb6-b566-703043171a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.049819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb40ae-5c37-4cb6-b566-703043171a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.051832 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eb40ae-5c37-4cb6-b566-703043171a13 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.072669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1066/2145] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:01.080561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0928a-9e27-4b59-bf4c-3d5ce751a741 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.081866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c8a0928a-9e27-4b59-bf4c-3d5ce751a741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:01.093450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1080/2146] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:16:01 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:16:01.094276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb40ae-5c37-4cb6-b566-703043171a13 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.100186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0928a-9e27-4b59-bf4c-3d5ce751a741 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.162540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99126b4f-21de-403f-8f9f-68d0e21e3559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.163526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99126b4f-21de-403f-8f9f-68d0e21e3559 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.164512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99126b4f-21de-403f-8f9f-68d0e21e3559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.164808 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99126b4f-21de-403f-8f9f-68d0e21e3559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.165568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99126b4f-21de-403f-8f9f-68d0e21e3559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.167699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99126b4f-21de-403f-8f9f-68d0e21e3559 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.196975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eb40ae-5c37-4cb6-b566-703043171a13 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.204470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1067/2147] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:16:01 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:16:01.205946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99126b4f-21de-403f-8f9f-68d0e21e3559 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.206416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492868e-b6c7-4281-92b4-c32ecf9a36f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.207111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492868e-b6c7-4281-92b4-c32ecf9a36f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.207783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492868e-b6c7-4281-92b4-c32ecf9a36f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.208607 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492868e-b6c7-4281-92b4-c32ecf9a36f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.209302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492868e-b6c7-4281-92b4-c32ecf9a36f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.212059 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492868e-b6c7-4281-92b4-c32ecf9a36f9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.246635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1081/2148] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:16:01 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:01.251819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492868e-b6c7-4281-92b4-c32ecf9a36f9 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.273885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492868e-b6c7-4281-92b4-c32ecf9a36f9 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.280890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:01.282085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:01.311621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99126b4f-21de-403f-8f9f-68d0e21e3559 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.318041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44442f9-098f-4908-87c8-ee1ba8ab7069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.318758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44442f9-098f-4908-87c8-ee1ba8ab7069 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.319052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44442f9-098f-4908-87c8-ee1ba8ab7069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.319369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44442f9-098f-4908-87c8-ee1ba8ab7069 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.320079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44442f9-098f-4908-87c8-ee1ba8ab7069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.322171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44442f9-098f-4908-87c8-ee1ba8ab7069 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.348466 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a5c96-2e33-4719-a9ed-0b3e6beb3b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.349603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a5c96-2e33-4719-a9ed-0b3e6beb3b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.350047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a5c96-2e33-4719-a9ed-0b3e6beb3b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.350483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a5c96-2e33-4719-a9ed-0b3e6beb3b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.350757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1068/2149] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:16:01 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:01.351605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a5c96-2e33-4719-a9ed-0b3e6beb3b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.353385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a5c96-2e33-4719-a9ed-0b3e6beb3b97 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.354524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44442f9-098f-4908-87c8-ee1ba8ab7069 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.366831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44442f9-098f-4908-87c8-ee1ba8ab7069 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.367433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1082/2150] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:01.371005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a5c96-2e33-4719-a9ed-0b3e6beb3b97 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.374582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b40a-1e2e-4cfd-a566-1e96b52b844a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.375677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b40a-1e2e-4cfd-a566-1e96b52b844a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.376189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b40a-1e2e-4cfd-a566-1e96b52b844a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.376659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239b40a-1e2e-4cfd-a566-1e96b52b844a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.377963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239b40a-1e2e-4cfd-a566-1e96b52b844a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.380872 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239b40a-1e2e-4cfd-a566-1e96b52b844a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.382901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a5c96-2e33-4719-a9ed-0b3e6beb3b97 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.389801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510aebf-4361-4381-a88f-deb7f25353bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.391670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510aebf-4361-4381-a88f-deb7f25353bd admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.391968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510aebf-4361-4381-a88f-deb7f25353bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.392372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510aebf-4361-4381-a88f-deb7f25353bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.393770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510aebf-4361-4381-a88f-deb7f25353bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.397447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1069/2151] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:01.397586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d510aebf-4361-4381-a88f-deb7f25353bd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.398444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b40a-1e2e-4cfd-a566-1e96b52b844a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.411407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239b40a-1e2e-4cfd-a566-1e96b52b844a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.412194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1083/2152] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:16:01.416265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510aebf-4361-4381-a88f-deb7f25353bd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.420019 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165a5c3-3ea2-4462-b3ba-2295bb7b09df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.420928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165a5c3-3ea2-4462-b3ba-2295bb7b09df admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.421213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165a5c3-3ea2-4462-b3ba-2295bb7b09df admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.421594 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3165a5c3-3ea2-4462-b3ba-2295bb7b09df admin admin] PATH_INFO: `/v3/projects/444ad00dc4084723bced21a6385809b1/users/2744765250db4c77bdd34fd4d5bf1759/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.426648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510aebf-4361-4381-a88f-deb7f25353bd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.431447 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759360c7-32d9-47eb-ad49-286867c91caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.432353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759360c7-32d9-47eb-ad49-286867c91caf admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.432638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759360c7-32d9-47eb-ad49-286867c91caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.432967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759360c7-32d9-47eb-ad49-286867c91caf admin admin] PATH_INFO: `/v3/projects/1c8a632cde9c4f0d97e2acaece1e2e37/users/91b0795e16f54d0a815f43fba605873c/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.445901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165a5c3-3ea2-4462-b3ba-2295bb7b09df admin admin] RBAC: Authorizing `identity:create_grant(project_id=444ad00dc4084723bced21a6385809b1, user_id=2744765250db4c77bdd34fd4d5bf1759, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.450518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3165a5c3-3ea2-4462-b3ba-2295bb7b09df admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.450816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759360c7-32d9-47eb-ad49-286867c91caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c8a632cde9c4f0d97e2acaece1e2e37, user_id=91b0795e16f54d0a815f43fba605873c, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.454083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759360c7-32d9-47eb-ad49-286867c91caf admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.472457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1070/2153] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:01 2022] PUT /identity/v3/projects/444ad00dc4084723bced21a6385809b1/users/2744765250db4c77bdd34fd4d5bf1759/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:01.475414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1084/2154] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:01 2022] PUT /identity/v3/projects/1c8a632cde9c4f0d97e2acaece1e2e37/users/91b0795e16f54d0a815f43fba605873c/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:01.476559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165a5c3-3ea2-4462-b3ba-2295bb7b09df admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.486688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5af9bd-8410-4bd9-bfea-26f6cda1fd81 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.487118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5af9bd-8410-4bd9-bfea-26f6cda1fd81 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.487851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5af9bd-8410-4bd9-bfea-26f6cda1fd81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.524442 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6c5af9bd-8410-4bd9-bfea-26f6cda1fd81 None None] MFA Rules not processed for user `91b0795e16f54d0a815f43fba605873c`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:01.525805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6c5af9bd-8410-4bd9-bfea-26f6cda1fd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:01.606305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1085/2155] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:01.606519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3165a5c3-3ea2-4462-b3ba-2295bb7b09df admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.619184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5af9bd-8410-4bd9-bfea-26f6cda1fd81 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.621275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6c5af9bd-8410-4bd9-bfea-26f6cda1fd81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:01.695624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a10aac-f807-4b88-852a-82467fef75e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.696207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a10aac-f807-4b88-852a-82467fef75e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.696207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a10aac-f807-4b88-852a-82467fef75e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.696619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a10aac-f807-4b88-852a-82467fef75e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.697370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a10aac-f807-4b88-852a-82467fef75e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.699392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a10aac-f807-4b88-852a-82467fef75e6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.700163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:01.712304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1071/2156] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:01.716886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a10aac-f807-4b88-852a-82467fef75e6 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.721939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5af9bd-8410-4bd9-bfea-26f6cda1fd81 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.726577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9badd83-cf76-43b9-907b-f86730cdd0a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.727241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9badd83-cf76-43b9-907b-f86730cdd0a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.727703 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9badd83-cf76-43b9-907b-f86730cdd0a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.728113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9badd83-cf76-43b9-907b-f86730cdd0a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.730159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a10aac-f807-4b88-852a-82467fef75e6 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:01.737129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:01.738382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.738732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9badd83-cf76-43b9-907b-f86730cdd0a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.739117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.739583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] PATH_INFO: `/v3/projects/444ad00dc4084723bced21a6385809b1/users/2744765250db4c77bdd34fd4d5bf1759/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.740409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9badd83-cf76-43b9-907b-f86730cdd0a1 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.742178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444ad00dc4084723bced21a6385809b1, user_id=2744765250db4c77bdd34fd4d5bf1759, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:01.745136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:01.753734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1086/2157] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:01.766541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1072/2158] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:01 2022] PUT /identity/v3/projects/444ad00dc4084723bced21a6385809b1/users/2744765250db4c77bdd34fd4d5bf1759/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:16:01.768769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186399-cd24-4c89-88ef-8443ef3e64b0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:01.769137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186399-cd24-4c89-88ef-8443ef3e64b0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:01.769547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf186399-cd24-4c89-88ef-8443ef3e64b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:01.804102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-bf186399-cd24-4c89-88ef-8443ef3e64b0 None None] MFA Rules not processed for user `2744765250db4c77bdd34fd4d5bf1759`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:01.805395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-bf186399-cd24-4c89-88ef-8443ef3e64b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:01.898844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1087/2159] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:01.911861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:01.913685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:01.971207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:02.016149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd71e2e-df6d-43d0-8646-b2ac532f2610 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:02.021221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:02.022245 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:02.022575 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:02.022888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:02.035526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:02.125716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:02.128070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:02.141102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1073/2160] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:05.027660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf186399-cd24-4c89-88ef-8443ef3e64b0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:05.037249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf186399-cd24-4c89-88ef-8443ef3e64b0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:05.040737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ed4c0-fd71-4999-826e-0a5219a9f403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:05.041372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ed4c0-fd71-4999-826e-0a5219a9f403 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:05.041649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ed4c0-fd71-4999-826e-0a5219a9f403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:05.041921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ed4c0-fd71-4999-826e-0a5219a9f403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:05.102115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ed4c0-fd71-4999-826e-0a5219a9f403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:05.103645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ed4c0-fd71-4999-826e-0a5219a9f403 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:05.113265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1088/2161] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:05 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:05.645985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:05.728494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832c9d17-4a57-4683-a091-2d91882299d1 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:05.735183 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:05.736316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:05.737778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:05.738182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] PATH_INFO: `/v3/users/91b0795e16f54d0a815f43fba605873c` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:05.740214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=91b0795e16f54d0a815f43fba605873c)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:05.742578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:05.861615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91b0795e16f54d0a815f43fba605873c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:05.863080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91b0795e16f54d0a815f43fba605873c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:05.864001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91b0795e16f54d0a815f43fba605873c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:05.886935 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91b0795e16f54d0a815f43fba605873c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:05.897753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91b0795e16f54d0a815f43fba605873c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:05.927168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:05.938922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1074/2162] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:16:05 2022] DELETE /identity/v3/users/91b0795e16f54d0a815f43fba605873c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:05.943523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706e0d9-33f0-404f-a4bb-ed89e35c338c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:05.946206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706e0d9-33f0-404f-a4bb-ed89e35c338c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:05.946605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706e0d9-33f0-404f-a4bb-ed89e35c338c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:05.972554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-0706e0d9-33f0-404f-a4bb-ed89e35c338c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:05.973793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0706e0d9-33f0-404f-a4bb-ed89e35c338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:06.065314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:06.067066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:06.076279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1089/2163] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:06.137019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706e0d9-33f0-404f-a4bb-ed89e35c338c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:06.138916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0706e0d9-33f0-404f-a4bb-ed89e35c338c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:06.149093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:06.248441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c199e-29d7-427a-b86b-38bc81691d4f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:06.255138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8e3b37-f0ab-45bb-b51b-b2fa1397ca61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:06.256101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3b37-f0ab-45bb-b51b-b2fa1397ca61 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:06.256388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3b37-f0ab-45bb-b51b-b2fa1397ca61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:06.256661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8e3b37-f0ab-45bb-b51b-b2fa1397ca61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:06.263094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5f8e3b37-f0ab-45bb-b51b-b2fa1397ca61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:06.312713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706e0d9-33f0-404f-a4bb-ed89e35c338c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:06.332237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:06.332913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:06.333197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:06.333471 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:06.343317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:06.344885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:06.368064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e3b37-f0ab-45bb-b51b-b2fa1397ca61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:06.369483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8e3b37-f0ab-45bb-b51b-b2fa1397ca61 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:06.372747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1090/2164] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:06.385449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1075/2165] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:06.762590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:06.764174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:06.783983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c957f-24c5-4026-bb51-59634b59da8f service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:06.789330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:06.790302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:06.790585 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:06.790858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] PATH_INFO: `/v3/projects/1c8a632cde9c4f0d97e2acaece1e2e37` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:06.792260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c8a632cde9c4f0d97e2acaece1e2e37)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:06.795075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:06.850553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c8a632cde9c4f0d97e2acaece1e2e37', 'request_id': 'req-dd50a206-65bc-4c36-a62d-556044f75bbf'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:06.856242 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c8a632cde9c4f0d97e2acaece1e2e37', 'request_id': 'req-dd50a206-65bc-4c36-a62d-556044f75bbf'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:06.863142 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c8a632cde9c4f0d97e2acaece1e2e37', 'request_id': 'req-dd50a206-65bc-4c36-a62d-556044f75bbf'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:06.876112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] The token cache is being invalidate because project 1c8a632cde9c4f0d97e2acaece1e2e37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:16:06.876351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:06.876969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-dd50a206-65bc-4c36-a62d-556044f75bbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:06.878660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1091/2166] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:16:06 2022] DELETE /identity/v3/projects/1c8a632cde9c4f0d97e2acaece1e2e37 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:06.883549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c9c4-6df6-41f2-9e87-5cfd65efeb51 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:06.884062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c9c4-6df6-41f2-9e87-5cfd65efeb51 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:06.884172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b847c9c4-6df6-41f2-9e87-5cfd65efeb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:06.903435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b847c9c4-6df6-41f2-9e87-5cfd65efeb51 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:06.904653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b847c9c4-6df6-41f2-9e87-5cfd65efeb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:06.963466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1076/2167] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:06.971117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bb21-35c2-4184-bf54-9b313e18dc61 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:06.971446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bb21-35c2-4184-bf54-9b313e18dc61 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:06.971724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f822bb21-35c2-4184-bf54-9b313e18dc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:06.992920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f822bb21-35c2-4184-bf54-9b313e18dc61 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:06.994212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f822bb21-35c2-4184-bf54-9b313e18dc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:07.003854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1092/2168] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:07.008423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847c9c4-6df6-41f2-9e87-5cfd65efeb51 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:07.019182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b847c9c4-6df6-41f2-9e87-5cfd65efeb51 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:07.024361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:07.025080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:07.025362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:07.025636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:07.026470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:16:07.026793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:07.028513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:07.040552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1077/2169] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:16:07.045002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822bb21-35c2-4184-bf54-9b313e18dc61 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:07.054482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f822bb21-35c2-4184-bf54-9b313e18dc61 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:07.062108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fcc74-854b-4cd9-bd4b-b92dc284c521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:07.062780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fcc74-854b-4cd9-bd4b-b92dc284c521 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:07.063084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fcc74-854b-4cd9-bd4b-b92dc284c521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:07.063396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150fcc74-854b-4cd9-bd4b-b92dc284c521 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:07.064263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150fcc74-854b-4cd9-bd4b-b92dc284c521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:07.066228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150fcc74-854b-4cd9-bd4b-b92dc284c521 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:07.095675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1093/2170] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:16:07 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:07.100677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:07.114514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b47423-81ea-42fb-9106-76f89266dec6 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:07.151904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:07.176322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395df38-0ce9-4748-9a91-02fecfb3b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:07.177002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395df38-0ce9-4748-9a91-02fecfb3b575 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:07.177291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395df38-0ce9-4748-9a91-02fecfb3b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:07.177561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5395df38-0ce9-4748-9a91-02fecfb3b575 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:07.178260 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395df38-0ce9-4748-9a91-02fecfb3b575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:07.180292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5395df38-0ce9-4748-9a91-02fecfb3b575 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:07.203699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1078/2171] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:16:07 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:07.207898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fcc74-854b-4cd9-bd4b-b92dc284c521 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:07.218305 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150fcc74-854b-4cd9-bd4b-b92dc284c521 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:07.223514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377ca5-31db-4b8b-a602-87c27c52ba2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:07.224180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377ca5-31db-4b8b-a602-87c27c52ba2f admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:07.224461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377ca5-31db-4b8b-a602-87c27c52ba2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:07.224737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377ca5-31db-4b8b-a602-87c27c52ba2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:07.225476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03377ca5-31db-4b8b-a602-87c27c52ba2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:07.227086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03377ca5-31db-4b8b-a602-87c27c52ba2f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:07.240754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1094/2172] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:07.242161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395df38-0ce9-4748-9a91-02fecfb3b575 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:07.252992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5395df38-0ce9-4748-9a91-02fecfb3b575 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:07.259163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f602563f-60df-4b63-8c4a-3c3b6ec92db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:07.259859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602563f-60df-4b63-8c4a-3c3b6ec92db3 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:07.260136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602563f-60df-4b63-8c4a-3c3b6ec92db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:07.260403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602563f-60df-4b63-8c4a-3c3b6ec92db3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:07.261145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602563f-60df-4b63-8c4a-3c3b6ec92db3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:07.262815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f602563f-60df-4b63-8c4a-3c3b6ec92db3 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:07.276995 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1079/2173] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:07.278370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377ca5-31db-4b8b-a602-87c27c52ba2f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:07.289132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377ca5-31db-4b8b-a602-87c27c52ba2f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:07.297705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:07.298769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:07.299051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:07.299358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] PATH_INFO: `/v3/projects/112d37976fba4b6dbdc27b9033f5a293/users/7530fd92f56e43ca95efc21cd9c49414/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:07.314595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] RBAC: Authorizing `identity:create_grant(project_id=112d37976fba4b6dbdc27b9033f5a293, user_id=7530fd92f56e43ca95efc21cd9c49414, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:07.317312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:07.331935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1095/2174] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:07 2022] PUT /identity/v3/projects/112d37976fba4b6dbdc27b9033f5a293/users/7530fd92f56e43ca95efc21cd9c49414/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:07.343038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f477c-af6f-4830-8393-28c4ce5c56f2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:07.343361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f477c-af6f-4830-8393-28c4ce5c56f2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:07.343815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599f477c-af6f-4830-8393-28c4ce5c56f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:07.376690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-599f477c-af6f-4830-8393-28c4ce5c56f2 None None] MFA Rules not processed for user `7530fd92f56e43ca95efc21cd9c49414`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:07.377906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-599f477c-af6f-4830-8393-28c4ce5c56f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:07.453205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1080/2175] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:07.466420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:07.468624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:07.515885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:07.551204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5c16a1-7437-436a-b3ec-8fb16a108aef admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:07.555164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:07.555722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:07.556012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:07.556289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:07.565920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:07.567434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:07.580608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1096/2176] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:08.347150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e51388-e92e-4dc2-b6fc-c216f2d3a581 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:08.347688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e51388-e92e-4dc2-b6fc-c216f2d3a581 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:08.347796 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e51388-e92e-4dc2-b6fc-c216f2d3a581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:08.403214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-89e51388-e92e-4dc2-b6fc-c216f2d3a581 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:08.404511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-89e51388-e92e-4dc2-b6fc-c216f2d3a581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:08.476526 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1081/2177] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:16:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:08.525157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:08.526765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:08.647528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb90052e-c643-45c0-bfa5-fa58abc348f0 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:08.653608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab5e3c-c744-488e-9504-302772f6d013 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:08.654295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab5e3c-c744-488e-9504-302772f6d013 service swift] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:08.654624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab5e3c-c744-488e-9504-302772f6d013 service swift] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:08.654947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab5e3c-c744-488e-9504-302772f6d013 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:08.664560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab5e3c-c744-488e-9504-302772f6d013 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:08.666683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab5e3c-c744-488e-9504-302772f6d013 service swift] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:08.677384 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1097/2178] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:16:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:16:09.721199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e51388-e92e-4dc2-b6fc-c216f2d3a581 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:09.722786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-89e51388-e92e-4dc2-b6fc-c216f2d3a581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:09.756679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:09.853155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e51388-e92e-4dc2-b6fc-c216f2d3a581 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:09.859333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:09.860226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:09.860494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:09.860947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] PATH_INFO: `/v3/users/ff11bb4f17574ef397c1d7060eabf628` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:09.862869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff11bb4f17574ef397c1d7060eabf628)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:09.868017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:09.954030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff11bb4f17574ef397c1d7060eabf628'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:09.955051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff11bb4f17574ef397c1d7060eabf628'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:09.955980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff11bb4f17574ef397c1d7060eabf628'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:09.969484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff11bb4f17574ef397c1d7060eabf628'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:09.980886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff11bb4f17574ef397c1d7060eabf628'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.036205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1082/2179] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:16:09 2022] DELETE /identity/v3/users/ff11bb4f17574ef397c1d7060eabf628 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:10.040941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd67c-203a-4497-adce-56c57c919c0b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:10.041808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd67c-203a-4497-adce-56c57c919c0b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:10.042266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cd67c-203a-4497-adce-56c57c919c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:10.080481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b35cd67c-203a-4497-adce-56c57c919c0b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:10.081734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b35cd67c-203a-4497-adce-56c57c919c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:10.189590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1098/2180] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:10.205519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:10.209924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:10.324951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c5f73-d26f-46ff-afae-063f0f47b25f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:10.328861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:10.330296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:10.330763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:10.331215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:10.343075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:10.344586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:10.356099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1083/2181] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:16:10.680973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cd67c-203a-4497-adce-56c57c919c0b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:10.683401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b35cd67c-203a-4497-adce-56c57c919c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:10.698502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cd67c-203a-4497-adce-56c57c919c0b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:10.704792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:10.706185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:10.706707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:10.707487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] PATH_INFO: `/v3/users/1f1b74fb31e642eca02d095a7292aacd` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:10.709731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1b74fb31e642eca02d095a7292aacd)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:10.713591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:10.747535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:10.750219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:10.792728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:10.798842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1ca00-e8a4-4cdb-a4e9-156b05ddf25f service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:10.802960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1b74fb31e642eca02d095a7292aacd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.806708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:10.807714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:10.808271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:10.808755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] PATH_INFO: `/v3/projects/bbfb0518f0604a5191f8721b1cb692c4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:10.810015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbfb0518f0604a5191f8721b1cb692c4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:10.812184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:10.812421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1b74fb31e642eca02d095a7292aacd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.813118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1b74fb31e642eca02d095a7292aacd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.827120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1b74fb31e642eca02d095a7292aacd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.827647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-fdae93c0-accc-49b8-8929-f038d755c9ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1b74fb31e642eca02d095a7292aacd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.856145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1099/2182] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:16:10 2022] DELETE /identity/v3/users/1f1b74fb31e642eca02d095a7292aacd => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:16:10.858073 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7c8df-b738-4fb9-bd17-4ac7f48fa58b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:10.858356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7c8df-b738-4fb9-bd17-4ac7f48fa58b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:10.858635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7c8df-b738-4fb9-bd17-4ac7f48fa58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:10.867511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:10.890834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbfb0518f0604a5191f8721b1cb692c4', 'request_id': 'req-0e29f326-6528-43d9-8992-d97ecc65ca31'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.895298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d0a7c8df-b738-4fb9-bd17-4ac7f48fa58b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:10.896686 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d0a7c8df-b738-4fb9-bd17-4ac7f48fa58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:10.908010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbfb0518f0604a5191f8721b1cb692c4', 'request_id': 'req-0e29f326-6528-43d9-8992-d97ecc65ca31'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.918552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbfb0518f0604a5191f8721b1cb692c4', 'request_id': 'req-0e29f326-6528-43d9-8992-d97ecc65ca31'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.940787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] The token cache is being invalidate because project bbfb0518f0604a5191f8721b1cb692c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:16:10.941186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.942532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0e29f326-6528-43d9-8992-d97ecc65ca31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:10.952449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1084/2183] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:16:10 2022] DELETE /identity/v3/projects/bbfb0518f0604a5191f8721b1cb692c4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:10.954530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062370a9-fb99-45a8-a853-1877ae8c50fd None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:10.955316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062370a9-fb99-45a8-a853-1877ae8c50fd None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:10.955553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062370a9-fb99-45a8-a853-1877ae8c50fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:10.980650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-062370a9-fb99-45a8-a853-1877ae8c50fd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:10.982060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-062370a9-fb99-45a8-a853-1877ae8c50fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:10.994206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1100/2184] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:11.006837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7c8df-b738-4fb9-bd17-4ac7f48fa58b None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.007056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d0a7c8df-b738-4fb9-bd17-4ac7f48fa58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:11.079313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1085/2185] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:11.087072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac0e64-f082-4e9c-8336-44cd09de0133 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.087389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac0e64-f082-4e9c-8336-44cd09de0133 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.088797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac0e64-f082-4e9c-8336-44cd09de0133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.103571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7c8df-b738-4fb9-bd17-4ac7f48fa58b None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:11.109857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbf07-2dd8-4c81-a4f0-f8b109110d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:11.111013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbf07-2dd8-4c81-a4f0-f8b109110d11 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.111481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbf07-2dd8-4c81-a4f0-f8b109110d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.111912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cbf07-2dd8-4c81-a4f0-f8b109110d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.123200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-08ac0e64-f082-4e9c-8336-44cd09de0133 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:11.123200 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-08ac0e64-f082-4e9c-8336-44cd09de0133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:11.126829 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1086/2186] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:11.130821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac0e64-f082-4e9c-8336-44cd09de0133 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.133776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cbf07-2dd8-4c81-a4f0-f8b109110d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:11.135511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cbf07-2dd8-4c81-a4f0-f8b109110d11 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:11.140437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac0e64-f082-4e9c-8336-44cd09de0133 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:11.145289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:11.145957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.146242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.146524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.147804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:16:11.148073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:11.149750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:11.152241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1101/2187] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:11.165920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1087/2188] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:16:11.172526 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbf07-2dd8-4c81-a4f0-f8b109110d11 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.188999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cbf07-2dd8-4c81-a4f0-f8b109110d11 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:11.196746 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd53d2-8506-4fe5-834a-516f2ea086c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:11.197682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd53d2-8506-4fe5-834a-516f2ea086c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.197961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd53d2-8506-4fe5-834a-516f2ea086c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.198235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadd53d2-8506-4fe5-834a-516f2ea086c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.199002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd53d2-8506-4fe5-834a-516f2ea086c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:11.201551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadd53d2-8506-4fe5-834a-516f2ea086c2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:11.319007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1102/2189] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:16:11 2022] POST /identity/v3/projects => generated 391 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:11.331644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.364983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293bd01-8032-4722-8cc1-5e6ea604d083 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:11.503243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45b699-51cb-496e-855a-9eb90852c83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:11.504380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45b699-51cb-496e-855a-9eb90852c83b admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.504951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45b699-51cb-496e-855a-9eb90852c83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.505378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f45b699-51cb-496e-855a-9eb90852c83b admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.506225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45b699-51cb-496e-855a-9eb90852c83b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:11.509447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f45b699-51cb-496e-855a-9eb90852c83b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:11.541983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1088/2190] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:16:11 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:11.546710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd53d2-8506-4fe5-834a-516f2ea086c2 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.562328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadd53d2-8506-4fe5-834a-516f2ea086c2 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:11.576074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:11.577022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.577511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.578002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.578973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:11.580870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:11.601950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1103/2191] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:16:11.607320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45b699-51cb-496e-855a-9eb90852c83b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.621276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f45b699-51cb-496e-855a-9eb90852c83b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:11.631415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0b1c6-b8cb-454d-b882-b5e6f3815d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:11.632549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0b1c6-b8cb-454d-b882-b5e6f3815d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.633106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0b1c6-b8cb-454d-b882-b5e6f3815d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.633795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e0b1c6-b8cb-454d-b882-b5e6f3815d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.634874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0b1c6-b8cb-454d-b882-b5e6f3815d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:11.636881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e0b1c6-b8cb-454d-b882-b5e6f3815d52 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:11.651937 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.653911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:11.668724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1089/2192] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:11.676134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0b1c6-b8cb-454d-b882-b5e6f3815d52 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.690812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e0b1c6-b8cb-454d-b882-b5e6f3815d52 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:11.698096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a14c61-03be-4d74-8aa6-845f971409d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:11.698096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a14c61-03be-4d74-8aa6-845f971409d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.698096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a14c61-03be-4d74-8aa6-845f971409d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.698096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a14c61-03be-4d74-8aa6-845f971409d2 admin admin] PATH_INFO: `/v3/projects/b72285aa2fec4027ba1af58764b65f39/users/3ada491e18284579810719410f101e75/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.709336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63055834-e820-4021-afc1-f2290367fd64 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:11.718640 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:11.718876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a14c61-03be-4d74-8aa6-845f971409d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b72285aa2fec4027ba1af58764b65f39, user_id=3ada491e18284579810719410f101e75, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:11.719573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.719856 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.720130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] PATH_INFO: `/v3/projects/9393be3d2e22466f97efd28482358a7b` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.722048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9393be3d2e22466f97efd28482358a7b)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:11.722363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a14c61-03be-4d74-8aa6-845f971409d2 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:11.724431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:11.739392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:11.752279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:11.758685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1090/2193] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:11 2022] PUT /identity/v3/projects/b72285aa2fec4027ba1af58764b65f39/users/3ada491e18284579810719410f101e75/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:11.783941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ca6d-3ba1-4a29-91fa-2ec37ffe46e8 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:11.784640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ca6d-3ba1-4a29-91fa-2ec37ffe46e8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:11.784938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4ca6d-3ba1-4a29-91fa-2ec37ffe46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:11.820884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9393be3d2e22466f97efd28482358a7b', 'request_id': 'req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:11.833863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9393be3d2e22466f97efd28482358a7b', 'request_id': 'req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:11.835641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4ef4ca6d-3ba1-4a29-91fa-2ec37ffe46e8 None None] MFA Rules not processed for user `3ada491e18284579810719410f101e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:11.837378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4ef4ca6d-3ba1-4a29-91fa-2ec37ffe46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:11.842432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9393be3d2e22466f97efd28482358a7b', 'request_id': 'req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:11.867849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] The token cache is being invalidate because project 9393be3d2e22466f97efd28482358a7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:16:11.868453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:11.869186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:11.871458 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1104/2194] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:16:11 2022] DELETE /identity/v3/projects/9393be3d2e22466f97efd28482358a7b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:11.875716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.877516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:11.968182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1091/2195] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:11.979762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4ca6d-3ba1-4a29-91fa-2ec37ffe46e8 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:11.985170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4ef4ca6d-3ba1-4a29-91fa-2ec37ffe46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:12.040540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1f4a7-0494-4a95-93ba-a0aa2a239f36 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:12.047863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:12.049174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:12.049653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:12.050137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] PATH_INFO: `/v3/users/15f43038911f4ddfadf2ded599f038bf` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:12.052228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=15f43038911f4ddfadf2ded599f038bf)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:12.054381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:12.107343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4ca6d-3ba1-4a29-91fa-2ec37ffe46e8 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:12.112908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d291664-e13e-4722-88cf-e77a1062f0ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:12.115316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d291664-e13e-4722-88cf-e77a1062f0ba service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:12.115982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d291664-e13e-4722-88cf-e77a1062f0ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:12.116563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d291664-e13e-4722-88cf-e77a1062f0ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:12.133863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d291664-e13e-4722-88cf-e77a1062f0ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:12.136124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15f43038911f4ddfadf2ded599f038bf'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.136811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d291664-e13e-4722-88cf-e77a1062f0ba service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:12.149715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15f43038911f4ddfadf2ded599f038bf'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.150978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15f43038911f4ddfadf2ded599f038bf'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.163824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15f43038911f4ddfadf2ded599f038bf'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.165177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15f43038911f4ddfadf2ded599f038bf'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.183644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1092/2196] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:16:12.190278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1105/2197] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:16:11 2022] DELETE /identity/v3/users/15f43038911f4ddfadf2ded599f038bf => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:12.585318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d291664-e13e-4722-88cf-e77a1062f0ba service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:12.616390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d291664-e13e-4722-88cf-e77a1062f0ba service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:12.720132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:12.722589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:12.722589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:12.722589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] PATH_INFO: `/v3/projects/445ed1dff2f14ea0a299bdaf2cde31a7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:12.724093 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=445ed1dff2f14ea0a299bdaf2cde31a7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:12.728815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:12.782062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:12.802002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '445ed1dff2f14ea0a299bdaf2cde31a7', 'request_id': 'req-3f1fdbfc-b46b-47b4-b3ed-763535352060'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.813525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '445ed1dff2f14ea0a299bdaf2cde31a7', 'request_id': 'req-3f1fdbfc-b46b-47b4-b3ed-763535352060'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.828141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '445ed1dff2f14ea0a299bdaf2cde31a7', 'request_id': 'req-3f1fdbfc-b46b-47b4-b3ed-763535352060'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.853383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] The token cache is being invalidate because project 445ed1dff2f14ea0a299bdaf2cde31a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:16:12.853383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.853383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:12.857642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1093/2198] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:16:12 2022] DELETE /identity/v3/projects/445ed1dff2f14ea0a299bdaf2cde31a7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:12.858903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:12.860429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:12.887729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625fe9ea-0c2e-4a75-ae1f-4786a09039ca admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:12.915362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:12.965309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:12.966222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:12.966502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:12.966774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] PATH_INFO: `/v3/users/2744765250db4c77bdd34fd4d5bf1759` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:12.968286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2744765250db4c77bdd34fd4d5bf1759)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:12.970455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:13.055724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2744765250db4c77bdd34fd4d5bf1759'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.063133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2744765250db4c77bdd34fd4d5bf1759'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.063876 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2744765250db4c77bdd34fd4d5bf1759'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.076061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2744765250db4c77bdd34fd4d5bf1759'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.077071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d999b53b-1c70-482c-9046-f9698001dbbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2744765250db4c77bdd34fd4d5bf1759'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.100063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1106/2199] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:16:12 2022] DELETE /identity/v3/users/2744765250db4c77bdd34fd4d5bf1759 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:13.479940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:13.483868 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:13.520864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1fdbfc-b46b-47b4-b3ed-763535352060 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:13.638828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:13.639933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:13.640383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:13.640809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] PATH_INFO: `/v3/projects/444ad00dc4084723bced21a6385809b1` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:13.642185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444ad00dc4084723bced21a6385809b1)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:13.644445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:13.708471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444ad00dc4084723bced21a6385809b1', 'request_id': 'req-4ba1d385-727a-4233-8c07-91886ddfb873'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.728505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444ad00dc4084723bced21a6385809b1', 'request_id': 'req-4ba1d385-727a-4233-8c07-91886ddfb873'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.735213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444ad00dc4084723bced21a6385809b1', 'request_id': 'req-4ba1d385-727a-4233-8c07-91886ddfb873'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.758205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] The token cache is being invalidate because project 444ad00dc4084723bced21a6385809b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:16:13.758786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.760201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4ba1d385-727a-4233-8c07-91886ddfb873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:13.762528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1094/2200] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:16:13 2022] DELETE /identity/v3/projects/444ad00dc4084723bced21a6385809b1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:13.767891 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97299898-2612-4ce1-be36-a8d12e08b81e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:13.769047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97299898-2612-4ce1-be36-a8d12e08b81e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:13.769615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97299898-2612-4ce1-be36-a8d12e08b81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:13.830369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-97299898-2612-4ce1-be36-a8d12e08b81e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:13.831395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-97299898-2612-4ce1-be36-a8d12e08b81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:13.939594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1107/2201] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:13.955469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1229a03-8aa6-40bf-adc5-f825ea521ea1 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:13.955988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1229a03-8aa6-40bf-adc5-f825ea521ea1 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:13.956088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1229a03-8aa6-40bf-adc5-f825ea521ea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:13.982280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c1229a03-8aa6-40bf-adc5-f825ea521ea1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:13.983489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c1229a03-8aa6-40bf-adc5-f825ea521ea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:14.000992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1095/2202] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:14.002307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97299898-2612-4ce1-be36-a8d12e08b81e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:14.020533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97299898-2612-4ce1-be36-a8d12e08b81e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:14.029844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:14.034459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:14.035561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:14.037924 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:14.039007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:16:14.039557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:14.043815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:14.060317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1108/2203] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:16:14.064820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1229a03-8aa6-40bf-adc5-f825ea521ea1 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:14.088506 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1229a03-8aa6-40bf-adc5-f825ea521ea1 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:14.104344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5ac04-b1a6-4628-8879-68fbc7b59cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:14.105264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5ac04-b1a6-4628-8879-68fbc7b59cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:14.105581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5ac04-b1a6-4628-8879-68fbc7b59cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:14.105859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5ac04-b1a6-4628-8879-68fbc7b59cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:14.106621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5ac04-b1a6-4628-8879-68fbc7b59cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:14.109893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5ac04-b1a6-4628-8879-68fbc7b59cb2 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:14.166107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1096/2204] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:16:14 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:14.167369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:14.192121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2a2e36-fa73-4fb3-9a9d-a457ab62420c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:14.293077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:14.346509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ed21-8ad4-45a2-a285-ca0d002610ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:14.347537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ed21-8ad4-45a2-a285-ca0d002610ad admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:14.347814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ed21-8ad4-45a2-a285-ca0d002610ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:14.348235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd15ed21-8ad4-45a2-a285-ca0d002610ad admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:14.349241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15ed21-8ad4-45a2-a285-ca0d002610ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:14.351547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd15ed21-8ad4-45a2-a285-ca0d002610ad admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:14.380726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1109/2205] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:16:14 2022] POST /identity/v3/users => generated 289 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:14.384882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5ac04-b1a6-4628-8879-68fbc7b59cb2 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:14.397408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5ac04-b1a6-4628-8879-68fbc7b59cb2 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:14.402334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcd84-0312-4024-a004-4f5ebeaa22ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:14.403237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcd84-0312-4024-a004-4f5ebeaa22ae admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:14.403644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcd84-0312-4024-a004-4f5ebeaa22ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:14.403992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dbcd84-0312-4024-a004-4f5ebeaa22ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:14.404831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbcd84-0312-4024-a004-4f5ebeaa22ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:14.406546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dbcd84-0312-4024-a004-4f5ebeaa22ae admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:14.419298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1097/2206] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:14.423797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ed21-8ad4-45a2-a285-ca0d002610ad admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:14.435781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd15ed21-8ad4-45a2-a285-ca0d002610ad admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:14.444421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6441a3-90b3-4409-9486-8d8a360adf51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:14.445102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6441a3-90b3-4409-9486-8d8a360adf51 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:14.445383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6441a3-90b3-4409-9486-8d8a360adf51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:14.445654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6441a3-90b3-4409-9486-8d8a360adf51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:14.446392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6441a3-90b3-4409-9486-8d8a360adf51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:14.448188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6441a3-90b3-4409-9486-8d8a360adf51 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:14.462236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1110/2207] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:14.466552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcd84-0312-4024-a004-4f5ebeaa22ae admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:14.482338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dbcd84-0312-4024-a004-4f5ebeaa22ae admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:14.487447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:14.488347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:14.488628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:14.488906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] PATH_INFO: `/v3/projects/f26ac8782fe0407cb6f146ba70e74246/users/bc0307909d614a1f97f006dc6d091cd4/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:14.500469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:14.510079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f26ac8782fe0407cb6f146ba70e74246, user_id=bc0307909d614a1f97f006dc6d091cd4, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:14.513086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:14.534296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1098/2208] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:14 2022] PUT /identity/v3/projects/f26ac8782fe0407cb6f146ba70e74246/users/bc0307909d614a1f97f006dc6d091cd4/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:14.543252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755e6b-f188-482a-8ae6-c935fe52d85f None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:14.543463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755e6b-f188-482a-8ae6-c935fe52d85f None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:14.543744 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a755e6b-f188-482a-8ae6-c935fe52d85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:14.565262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8a755e6b-f188-482a-8ae6-c935fe52d85f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:14.566966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8a755e6b-f188-482a-8ae6-c935fe52d85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:14.658739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1111/2209] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:14.669474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:14.671709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:14.738451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32551a6c-abe0-4c7a-afda-dbf96f7d3f71 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:14.742326 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b17b0-4427-4317-bea5-90a14dc00331 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:14.743061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b17b0-4427-4317-bea5-90a14dc00331 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:14.743444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b17b0-4427-4317-bea5-90a14dc00331 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:14.743809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b17b0-4427-4317-bea5-90a14dc00331 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:14.758349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b17b0-4427-4317-bea5-90a14dc00331 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:14.760531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4b17b0-4427-4317-bea5-90a14dc00331 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:14.778532 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1099/2210] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:21.788594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5b41-da0a-45ec-96d7-80417b352305 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:21.789124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5b41-da0a-45ec-96d7-80417b352305 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:21.789222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0e5b41-da0a-45ec-96d7-80417b352305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:21.818569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3e0e5b41-da0a-45ec-96d7-80417b352305 None None] MFA Rules not processed for user `bc0307909d614a1f97f006dc6d091cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:21.819817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3e0e5b41-da0a-45ec-96d7-80417b352305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:21.890317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1112/2211] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:21.894707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b17b0-4427-4317-bea5-90a14dc00331 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:21.906045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b17b0-4427-4317-bea5-90a14dc00331 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:21.911068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98221d7-8e7a-42d8-b716-3fa74b5a88ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:21.911788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98221d7-8e7a-42d8-b716-3fa74b5a88ea admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:21.912074 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98221d7-8e7a-42d8-b716-3fa74b5a88ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:21.912495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98221d7-8e7a-42d8-b716-3fa74b5a88ea admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:21.913426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98221d7-8e7a-42d8-b716-3fa74b5a88ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:21.915439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98221d7-8e7a-42d8-b716-3fa74b5a88ea admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:21.947314 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1100/2212] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:16:21 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:16:21.950229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e5b41-da0a-45ec-96d7-80417b352305 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:21.961752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0e5b41-da0a-45ec-96d7-80417b352305 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:22.010488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:22.034704 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60895c-8a7d-46f0-831b-b19cc740f44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:22.035510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60895c-8a7d-46f0-831b-b19cc740f44d admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:22.035859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60895c-8a7d-46f0-831b-b19cc740f44d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:22.036201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f60895c-8a7d-46f0-831b-b19cc740f44d admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:22.036968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60895c-8a7d-46f0-831b-b19cc740f44d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:22.038968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f60895c-8a7d-46f0-831b-b19cc740f44d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:22.061836 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1113/2213] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:16:21 2022] POST /identity/v3/users => generated 294 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:16:22.065838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98221d7-8e7a-42d8-b716-3fa74b5a88ea admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:22.076571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98221d7-8e7a-42d8-b716-3fa74b5a88ea admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:22.081217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb4dc7-b058-4518-895e-1c4a4c0c9a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:22.081902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4dc7-b058-4518-895e-1c4a4c0c9a4b admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:22.082181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4dc7-b058-4518-895e-1c4a4c0c9a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:22.082454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb4dc7-b058-4518-895e-1c4a4c0c9a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:22.083202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4dc7-b058-4518-895e-1c4a4c0c9a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:22.084874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fb4dc7-b058-4518-895e-1c4a4c0c9a4b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:22.096293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1101/2214] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:16:22.100413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60895c-8a7d-46f0-831b-b19cc740f44d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:22.110957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f60895c-8a7d-46f0-831b-b19cc740f44d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:22.116148 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d46ce-206d-4869-9885-e37d966b8af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:22.116879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d46ce-206d-4869-9885-e37d966b8af6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:22.117159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d46ce-206d-4869-9885-e37d966b8af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:22.117459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3d46ce-206d-4869-9885-e37d966b8af6 admin admin] PATH_INFO: `/v3/projects/d05f58af207e431eac6b187af266d58d/users/fddb4b4558dd47b9925995a565bf9fd3/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:22.134489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d46ce-206d-4869-9885-e37d966b8af6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05f58af207e431eac6b187af266d58d, user_id=fddb4b4558dd47b9925995a565bf9fd3, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:22.141208 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3d46ce-206d-4869-9885-e37d966b8af6 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:22.159158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1114/2215] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:22 2022] PUT /identity/v3/projects/d05f58af207e431eac6b187af266d58d/users/fddb4b4558dd47b9925995a565bf9fd3/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:16:22.162892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb4dc7-b058-4518-895e-1c4a4c0c9a4b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:22.174092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb4dc7-b058-4518-895e-1c4a4c0c9a4b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:22.240475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd315b6-9758-4c55-bc81-7827b66f5bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:22.241351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd315b6-9758-4c55-bc81-7827b66f5bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:22.241830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd315b6-9758-4c55-bc81-7827b66f5bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:22.245310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd315b6-9758-4c55-bc81-7827b66f5bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:22.246166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd315b6-9758-4c55-bc81-7827b66f5bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:22.247921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd315b6-9758-4c55-bc81-7827b66f5bd4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:22.259317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1102/2216] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:16:22.264326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d46ce-206d-4869-9885-e37d966b8af6 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:22.274857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3d46ce-206d-4869-9885-e37d966b8af6 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:22.279449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:22.280340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:22.280621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:22.280899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] PATH_INFO: `/v3/projects/d05f58af207e431eac6b187af266d58d/users/fddb4b4558dd47b9925995a565bf9fd3/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:22.282895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d05f58af207e431eac6b187af266d58d, user_id=fddb4b4558dd47b9925995a565bf9fd3, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:22.285573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:22.296812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1115/2217] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:16:22 2022] PUT /identity/v3/projects/d05f58af207e431eac6b187af266d58d/users/fddb4b4558dd47b9925995a565bf9fd3/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:16:28.086941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4d987-cfef-490c-af54-2185dc912984 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:28.087465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4d987-cfef-490c-af54-2185dc912984 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:28.087558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d4d987-cfef-490c-af54-2185dc912984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:28.114888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-75d4d987-cfef-490c-af54-2185dc912984 None None] MFA Rules not processed for user `fddb4b4558dd47b9925995a565bf9fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:28.116162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-75d4d987-cfef-490c-af54-2185dc912984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:28.157254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:28.187573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1103/2218] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:28.201745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:28.203783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:28.300047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba94c42-11a6-497f-af5c-9a3862f108a4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:28.306397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:28.307045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:28.307351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:28.307627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:28.365159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:28.366751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:28.376000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1116/2219] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:30.937392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4d987-cfef-490c-af54-2185dc912984 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:30.939776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-75d4d987-cfef-490c-af54-2185dc912984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:30.987366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:30.990705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:31.029642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:31.082979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d4d987-cfef-490c-af54-2185dc912984 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:31.101016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced8d02-c150-438e-89f5-b27ec2df75ea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:31.101853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8d02-c150-438e-89f5-b27ec2df75ea service placement] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:31.102364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8d02-c150-438e-89f5-b27ec2df75ea service placement] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:31.102638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced8d02-c150-438e-89f5-b27ec2df75ea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:31.105247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7ced8d02-c150-438e-89f5-b27ec2df75ea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:31.118946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced8d02-c150-438e-89f5-b27ec2df75ea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:31.120982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2ea71-b294-4d4c-ac1c-a31370c0e165 service nova] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:31.121965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced8d02-c150-438e-89f5-b27ec2df75ea service placement] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:31.126872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9ff224-76e5-44f1-a496-bd5eadc0ae5b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:31.127572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ff224-76e5-44f1-a496-bd5eadc0ae5b service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:31.127852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ff224-76e5-44f1-a496-bd5eadc0ae5b service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:31.128124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ff224-76e5-44f1-a496-bd5eadc0ae5b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:31.134494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1104/2220] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:31.138398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ff224-76e5-44f1-a496-bd5eadc0ae5b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:31.140154 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9ff224-76e5-44f1-a496-bd5eadc0ae5b service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:31.153307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1117/2221] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:16:33.188555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737b381-a68c-4a5c-bc14-e3647d69b4a8 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:33.189452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737b381-a68c-4a5c-bc14-e3647d69b4a8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:33.189452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737b381-a68c-4a5c-bc14-e3647d69b4a8 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:33.191155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1105/2222] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:16:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:16:33.201463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49aee1-bd76-4491-9966-9e3c1853c8fd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:33.201969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49aee1-bd76-4491-9966-9e3c1853c8fd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:33.202071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f49aee1-bd76-4491-9966-9e3c1853c8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:33.255231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-7f49aee1-bd76-4491-9966-9e3c1853c8fd None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:33.257374 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7f49aee1-bd76-4491-9966-9e3c1853c8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:33.282080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1118/2223] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:16:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:16:34.667097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737b381-a68c-4a5c-bc14-e3647d69b4a8 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:34.669365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7737b381-a68c-4a5c-bc14-e3647d69b4a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:34.741734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737b381-a68c-4a5c-bc14-e3647d69b4a8 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:34.749008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:34.749956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:34.750235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:34.750514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] PATH_INFO: `/v3/users/3ada491e18284579810719410f101e75` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:34.751838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ada491e18284579810719410f101e75)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:34.753865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:34.809953 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ada491e18284579810719410f101e75'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:34.811466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ada491e18284579810719410f101e75'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:34.812463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ada491e18284579810719410f101e75'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:34.815788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:34.824283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ada491e18284579810719410f101e75'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:34.836180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ada491e18284579810719410f101e75'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:34.856341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1106/2224] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:16:34 2022] DELETE /identity/v3/users/3ada491e18284579810719410f101e75 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:34.861977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df4e5b-31b2-42d0-8948-574625a85466 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:34.862338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df4e5b-31b2-42d0-8948-574625a85466 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:34.862637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df4e5b-31b2-42d0-8948-574625a85466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:34.880924 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-75df4e5b-31b2-42d0-8948-574625a85466 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:34.882112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-75df4e5b-31b2-42d0-8948-574625a85466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:34.943625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1119/2225] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:34.955848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:34.957566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:35.025847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49bbb68-327f-471d-967c-70f3b1282a39 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:35.029939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:35.030688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:35.030839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:35.031125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:35.041846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:35.043334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:35.053505 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1107/2226] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:35.328220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df4e5b-31b2-42d0-8948-574625a85466 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:35.329933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-75df4e5b-31b2-42d0-8948-574625a85466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:35.345349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df4e5b-31b2-42d0-8948-574625a85466 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:35.350349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:35.351501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:35.351984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:35.352347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] PATH_INFO: `/v3/projects/b72285aa2fec4027ba1af58764b65f39` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:35.354138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b72285aa2fec4027ba1af58764b65f39)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:35.356718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:35.406843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b72285aa2fec4027ba1af58764b65f39', 'request_id': 'req-63814994-4af4-43fa-a836-99c479956a10'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:35.412219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b72285aa2fec4027ba1af58764b65f39', 'request_id': 'req-63814994-4af4-43fa-a836-99c479956a10'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:35.418385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b72285aa2fec4027ba1af58764b65f39', 'request_id': 'req-63814994-4af4-43fa-a836-99c479956a10'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:35.432723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] The token cache is being invalidate because project b72285aa2fec4027ba1af58764b65f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:16:35.433009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:35.433815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63814994-4af4-43fa-a836-99c479956a10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:16:35.435452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1120/2227] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:16:35 2022] DELETE /identity/v3/projects/b72285aa2fec4027ba1af58764b65f39 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:16:45.048844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=3,keystone:SELECT=38 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:45.429514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:47.408379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:47.409962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:47.496394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0383d671-57d1-4ed6-8011-37a611ce005e service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:47.556266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4892f1dc-fb5b-489a-8a2b-869f3fdeb3aa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d25a45f9ee4ae58c6751a8781d12d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:47.556948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892f1dc-fb5b-489a-8a2b-869f3fdeb3aa service glance] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:47.557229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892f1dc-fb5b-489a-8a2b-869f3fdeb3aa service glance] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:47.557550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4892f1dc-fb5b-489a-8a2b-869f3fdeb3aa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:47.559554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4892f1dc-fb5b-489a-8a2b-869f3fdeb3aa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:47.630303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892f1dc-fb5b-489a-8a2b-869f3fdeb3aa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:47.631833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4892f1dc-fb5b-489a-8a2b-869f3fdeb3aa service glance] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:47.643072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1108/2228] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:47.719367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164586e-ec19-4c55-aeed-5c23e58c3b2d None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:47.719918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164586e-ec19-4c55-aeed-5c23e58c3b2d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:47.720197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0164586e-ec19-4c55-aeed-5c23e58c3b2d None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:47.721608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1121/2229] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:16:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:16:47.726861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed14665-63f0-4648-8e80-c540e47bb75c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:47.727153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed14665-63f0-4648-8e80-c540e47bb75c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:47.727462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed14665-63f0-4648-8e80-c540e47bb75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:47.745981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-bed14665-63f0-4648-8e80-c540e47bb75c None None] MFA Rules not processed for user `349e99511c7c4307b27dc90b01f2dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:47.747159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bed14665-63f0-4648-8e80-c540e47bb75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:47.781057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:16:47.801229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1109/2230] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:16:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:47.813739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164586e-ec19-4c55-aeed-5c23e58c3b2d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:47.815652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0164586e-ec19-4c55-aeed-5c23e58c3b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:47.829704 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0164586e-ec19-4c55-aeed-5c23e58c3b2d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:47.833248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a72eb-e431-4d7c-86b8-b5a6677c8513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:47.833879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a72eb-e431-4d7c-86b8-b5a6677c8513 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:47.834157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a72eb-e431-4d7c-86b8-b5a6677c8513 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:47.834432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a72eb-e431-4d7c-86b8-b5a6677c8513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:47.841780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a72eb-e431-4d7c-86b8-b5a6677c8513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:47.843202 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a72eb-e431-4d7c-86b8-b5a6677c8513 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:47.850955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1122/2231] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:47 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:47.934814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c34d-3f5e-482e-b06e-63140f1185f3 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:47.935188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c34d-3f5e-482e-b06e-63140f1185f3 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:47.935633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155c34d-3f5e-482e-b06e-63140f1185f3 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:47.937125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1110/2232] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:16:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:16:47.942765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d2b1b-6d20-44a8-a0e1-41da6518d607 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:47.943066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d2b1b-6d20-44a8-a0e1-41da6518d607 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:47.943409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90d2b1b-6d20-44a8-a0e1-41da6518d607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:47.964580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a90d2b1b-6d20-44a8-a0e1-41da6518d607 None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:47.965772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a90d2b1b-6d20-44a8-a0e1-41da6518d607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:47.979617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1123/2233] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:16:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:47.990305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155c34d-3f5e-482e-b06e-63140f1185f3 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:47.992176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6155c34d-3f5e-482e-b06e-63140f1185f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:48.006111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155c34d-3f5e-482e-b06e-63140f1185f3 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:48.009919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:48.010621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:48.010903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:48.011176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:48.021827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:48.023611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:48.033953 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1111/2234] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:16:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:16:50.702179 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5d53-b508-48a8-b00c-bb26215fe1c2 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:50.702718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5d53-b508-48a8-b00c-bb26215fe1c2 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:50.702718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ad5d53-b508-48a8-b00c-bb26215fe1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:50.728946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d0ad5d53-b508-48a8-b00c-bb26215fe1c2 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:50.730676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d0ad5d53-b508-48a8-b00c-bb26215fe1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:50.793309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1124/2235] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:16:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:16:50.805741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:16:50.807217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:50.879043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5220c7-ee01-4663-a2d3-bb9f5d882707 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:16:50.885341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:16:50.886003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:50.886243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:50.886555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:50.897981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:16:50.899569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:16:50.908383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1112/2236] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:16:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:16:53.464693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80862-15c5-475b-8e8b-462cc1ab5b49 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:16:53.465284 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80862-15c5-475b-8e8b-462cc1ab5b49 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:16:53.465284 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb80862-15c5-475b-8e8b-462cc1ab5b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:16:53.485141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-adb80862-15c5-475b-8e8b-462cc1ab5b49 None None] MFA Rules not processed for user `bc0307909d614a1f97f006dc6d091cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:16:53.486604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-adb80862-15c5-475b-8e8b-462cc1ab5b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:16:53.607599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1125/2237] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:16:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:17:00.549870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:17:00.551575 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:00.623235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e6fe11-59b3-42ba-b552-67a93a3ea0a1 service swift] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:17:00.654606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:17:00.655650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:00.656031 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:00.656416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] PATH_INFO: `/v3/users/7530fd92f56e43ca95efc21cd9c49414` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:00.657778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7530fd92f56e43ca95efc21cd9c49414)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:17:00.660054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:17:00.749272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7530fd92f56e43ca95efc21cd9c49414'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:00.749952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7530fd92f56e43ca95efc21cd9c49414'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:00.750574 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7530fd92f56e43ca95efc21cd9c49414'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:00.769408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7530fd92f56e43ca95efc21cd9c49414'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:00.782746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7530fd92f56e43ca95efc21cd9c49414'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:00.784613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:17:00.812771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1113/2238] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:17:00 2022] DELETE /identity/v3/users/7530fd92f56e43ca95efc21cd9c49414 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:17:00.817450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be80c0-3978-4472-9f4c-3b673db29192 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:00.817973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be80c0-3978-4472-9f4c-3b673db29192 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:00.818071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be80c0-3978-4472-9f4c-3b673db29192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:00.834964 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e6be80c0-3978-4472-9f4c-3b673db29192 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:17:00.836205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e6be80c0-3978-4472-9f4c-3b673db29192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:00.898294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1126/2239] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:17:00.907973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:17:00.909787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:00.974116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e219d2c1-a45e-4b82-9e65-35ef92895b3d admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:17:00.978619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62153c8d-6c59-4170-9bc9-9c230e2d53d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:17:00.979734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62153c8d-6c59-4170-9bc9-9c230e2d53d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:00.980203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62153c8d-6c59-4170-9bc9-9c230e2d53d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:00.980650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62153c8d-6c59-4170-9bc9-9c230e2d53d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:00.992995 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62153c8d-6c59-4170-9bc9-9c230e2d53d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:17:00.996196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62153c8d-6c59-4170-9bc9-9c230e2d53d7 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:17:01.008689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1114/2240] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:17:01.410648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be80c0-3978-4472-9f4c-3b673db29192 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:17:01.412282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e6be80c0-3978-4472-9f4c-3b673db29192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:01.425036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be80c0-3978-4472-9f4c-3b673db29192 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:17:01.431481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:17:01.432381 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:01.433174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:01.433174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] PATH_INFO: `/v3/projects/112d37976fba4b6dbdc27b9033f5a293` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:01.434186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=112d37976fba4b6dbdc27b9033f5a293)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:17:01.436254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:17:01.485112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:17:01.515452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112d37976fba4b6dbdc27b9033f5a293', 'request_id': 'req-f906e584-d0e2-4a08-957a-2667070bed6f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:01.521673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112d37976fba4b6dbdc27b9033f5a293', 'request_id': 'req-f906e584-d0e2-4a08-957a-2667070bed6f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:01.528672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112d37976fba4b6dbdc27b9033f5a293', 'request_id': 'req-f906e584-d0e2-4a08-957a-2667070bed6f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:01.545819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] The token cache is being invalidate because project 112d37976fba4b6dbdc27b9033f5a293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:17:01.546417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:01.547133 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:17:01.548935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1127/2241] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:17:01 2022] DELETE /identity/v3/projects/112d37976fba4b6dbdc27b9033f5a293 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:17:11.003662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=32,keystone:DELETE=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:17:11.541470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:17:20.590639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5389b8-318e-4359-8e23-ad4c27bd7e4f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:20.591106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5389b8-318e-4359-8e23-ad4c27bd7e4f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:20.591214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5389b8-318e-4359-8e23-ad4c27bd7e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:20.609770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-8e5389b8-318e-4359-8e23-ad4c27bd7e4f None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:17:20.610968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8e5389b8-318e-4359-8e23-ad4c27bd7e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:20.668257 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1115/2242] 10.222.0.69 () {58 vars in 967 bytes} [Fri May 20 09:17:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:17:20.691514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:17:20.693232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:20.764794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906e584-d0e2-4a08-957a-2667070bed6f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:17:20.771639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:17:20.772319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:20.772619 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:20.772893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:20.781503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:17:20.783118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:17:20.791332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1128/2243] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:17:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:17:24.379354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0527cb-ee67-4b46-85d9-7e77d82ccde6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:24.379856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0527cb-ee67-4b46-85d9-7e77d82ccde6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:24.379953 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0527cb-ee67-4b46-85d9-7e77d82ccde6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:24.403439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-aa0527cb-ee67-4b46-85d9-7e77d82ccde6 None None] MFA Rules not processed for user `ff944e4cf29b4297acc63b591d90daaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:17:24.404653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-aa0527cb-ee67-4b46-85d9-7e77d82ccde6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:24.463281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1116/2244] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:17:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:17:27.393369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:17:27.395439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:27.456713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae960f80-ae30-44f0-8d9e-0adaa4537c77 service swift] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:17:27.460446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2850fdba-1f28-42da-96b5-040d99ad6a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:17:27.461376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850fdba-1f28-42da-96b5-040d99ad6a2d service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:27.461376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850fdba-1f28-42da-96b5-040d99ad6a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:27.461663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2850fdba-1f28-42da-96b5-040d99ad6a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:27.463635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2850fdba-1f28-42da-96b5-040d99ad6a2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:27.529541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850fdba-1f28-42da-96b5-040d99ad6a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:17:27.530985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2850fdba-1f28-42da-96b5-040d99ad6a2d service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:17:27.540586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1129/2245] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:17:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:17:34.373758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f574f7-c551-4a5a-ab05-a89d6c892fcc None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:17:34.374265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f574f7-c551-4a5a-ab05-a89d6c892fcc None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:17:34.374265 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f574f7-c551-4a5a-ab05-a89d6c892fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:17:34.395371 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-68f574f7-c551-4a5a-ab05-a89d6c892fcc None None] MFA Rules not processed for user `992571802b7d426f923e0f0bc41e28de`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:17:34.396600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-68f574f7-c551-4a5a-ab05-a89d6c892fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:17:34.461524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1117/2246] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:17:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:17:37.536667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=82 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:17:44.453888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=75 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:18:08.819670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1419-4738-44e6-99d3-79b80efdda45 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:08.820233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1419-4738-44e6-99d3-79b80efdda45 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:08.820233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348f1419-4738-44e6-99d3-79b80efdda45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:08.838001 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-348f1419-4738-44e6-99d3-79b80efdda45 None None] MFA Rules not processed for user `a7e574dc676841dea1f23a06c333ff14`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:18:08.839218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-348f1419-4738-44e6-99d3-79b80efdda45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:08.896109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1130/2247] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:18:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:18:17.726885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f574f7-c551-4a5a-ab05-a89d6c892fcc None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:17.728642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-68f574f7-c551-4a5a-ab05-a89d6c892fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:17.741962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f574f7-c551-4a5a-ab05-a89d6c892fcc None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:17.746499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:17.747093 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:17.747435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:17.747708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:17.749650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:17.828224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:17.829666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:17.840600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1118/2248] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:18.889099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=25 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:18:21.615115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f1419-4738-44e6-99d3-79b80efdda45 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:21.616779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-348f1419-4738-44e6-99d3-79b80efdda45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:21.677632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348f1419-4738-44e6-99d3-79b80efdda45 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:21.682739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:21.682739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:21.682739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:21.683073 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:21.684959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:21.699171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:21.700632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:21.711478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1131/2249] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:25.507843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:25.509621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:25.525750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda1085d-7fb9-40dd-8cf3-dcef7d38c053 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:25.533710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:25.534416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:25.534728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:25.535028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:25.537315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:25.548065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:25.549683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:25.559063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1119/2250] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:31.707010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=26 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:18:35.554701 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=40 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:18:38.950077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:38.959601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6c44f-ba34-4deb-a5f8-de27b3c611c7 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:38.966648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed0c5fc-695f-4a4c-93e8-6cbd5930b734 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:38.967688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0c5fc-695f-4a4c-93e8-6cbd5930b734 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:38.968098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0c5fc-695f-4a4c-93e8-6cbd5930b734 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:38.968488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed0c5fc-695f-4a4c-93e8-6cbd5930b734 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:38.970888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0ed0c5fc-695f-4a4c-93e8-6cbd5930b734 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:39.042172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0c5fc-695f-4a4c-93e8-6cbd5930b734 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:39.044187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed0c5fc-695f-4a4c-93e8-6cbd5930b734 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:39.055752 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1132/2251] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:18:39.325682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:39.336673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22126ac-0d65-438a-9e4f-d5e9c42c1d17 service nova] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:39.342108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:39.343442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:39.343822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:39.344190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:39.357980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:39.380943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:39.382909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:39.397150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1120/2252] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:41.108799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1567f-9a53-42fa-bdd6-a9a41b6370f9 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:41.109370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1567f-9a53-42fa-bdd6-a9a41b6370f9 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:41.109370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1567f-9a53-42fa-bdd6-a9a41b6370f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:41.127943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-deb1567f-9a53-42fa-bdd6-a9a41b6370f9 None None] MFA Rules not processed for user `ff944e4cf29b4297acc63b591d90daaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:18:41.129273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-deb1567f-9a53-42fa-bdd6-a9a41b6370f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:41.136392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1133/2253] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:18:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:18:42.297318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:42.298903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:42.373010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c526e6-3f0c-4b31-a828-5598b35efa59 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:42.378726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:42.379519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:42.379825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:42.380104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:42.381923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:42.394348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:42.396004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:42.406068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1121/2254] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:42.496969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f6baf-b1c9-40e4-b592-f8c3cede31b0 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:42.497501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f6baf-b1c9-40e4-b592-f8c3cede31b0 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:42.497501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f6baf-b1c9-40e4-b592-f8c3cede31b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:42.515985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-7b5f6baf-b1c9-40e4-b592-f8c3cede31b0 None None] MFA Rules not processed for user `bfaaa367491745708334c5f35331bf76`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:18:42.517268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7b5f6baf-b1c9-40e4-b592-f8c3cede31b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:42.525101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1134/2255] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:18:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:18:44.485996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:44.498394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d9f56b-2734-44cb-bbe7-981f1c4abe3a service placement] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:44.502430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:44.504442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:44.504890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:44.505313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:44.507947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:44.519067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:44.521274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:44.541271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1122/2256] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:44.558741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f6baf-b1c9-40e4-b592-f8c3cede31b0 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:44.573061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f6baf-b1c9-40e4-b592-f8c3cede31b0 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:44.577553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:44.578424 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:44.578849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:44.579309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:44.581415 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:44.736343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:44.737907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:44.758709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1135/2257] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:50.102962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:50.118910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:50.137353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0655d-340f-4910-9f6e-8cc5dcc98a87 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:50.144936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:50.145835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:50.146130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:50.146413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:50.148703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:50.162579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:50.164294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:50.179924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1123/2258] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:50.196909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:50.198413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:50.210799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2244c20-08d6-4606-acde-8c7edc5239d8 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:50.215881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:50.216236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:50.216514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:50.216785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] PATH_INFO: `/v3/users/992571802b7d426f923e0f0bc41e28de` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:50.218294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] RBAC: Authorizing `identity:delete_user(user_id=992571802b7d426f923e0f0bc41e28de)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:50.220260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:50.281681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992571802b7d426f923e0f0bc41e28de'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.289455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992571802b7d426f923e0f0bc41e28de'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.290149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992571802b7d426f923e0f0bc41e28de'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.298486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992571802b7d426f923e0f0bc41e28de'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.299462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d2546a1d-39b9-4da6-a291-b8a8010a069a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992571802b7d426f923e0f0bc41e28de'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.307813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:18:50.319141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1136/2259] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:18:50 2022] DELETE /identity/v3/users/992571802b7d426f923e0f0bc41e28de => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:18:50.565589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:50.577819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f812276-2e16-49a2-94ed-6459ee73b113 service placement] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:50.641826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:50.642657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:50.642937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:50.643212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] PATH_INFO: `/v3/projects/ed0e52a9d6374123b57759435d43d7aa` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:50.644871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0e52a9d6374123b57759435d43d7aa)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:50.646850 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:50.696961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0e52a9d6374123b57759435d43d7aa', 'request_id': 'req-423cce63-d158-4ad5-8613-53ef591ca596'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.699416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:18:50.706256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0e52a9d6374123b57759435d43d7aa', 'request_id': 'req-423cce63-d158-4ad5-8613-53ef591ca596'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.713548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0e52a9d6374123b57759435d43d7aa', 'request_id': 'req-423cce63-d158-4ad5-8613-53ef591ca596'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.725687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] The token cache is being invalidate because project ed0e52a9d6374123b57759435d43d7aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:18:50.725973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.726613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-423cce63-d158-4ad5-8613-53ef591ca596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:18:50.728275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1124/2260] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:18:50 2022] DELETE /identity/v3/projects/ed0e52a9d6374123b57759435d43d7aa => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:18:50.733530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1d7b4-c18f-4a3e-9181-fb6125fa0c09 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:50.733940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1d7b4-c18f-4a3e-9181-fb6125fa0c09 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:50.734313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1d7b4-c18f-4a3e-9181-fb6125fa0c09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:50.753355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-09c1d7b4-c18f-4a3e-9181-fb6125fa0c09 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:18:50.754611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-09c1d7b4-c18f-4a3e-9181-fb6125fa0c09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:50.822943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1137/2261] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:18:50.830689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7be501-5400-48d5-afd5-f60b7d95f279 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:50.830975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7be501-5400-48d5-afd5-f60b7d95f279 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:50.831250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7be501-5400-48d5-afd5-f60b7d95f279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:50.849226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-2f7be501-5400-48d5-afd5-f60b7d95f279 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:18:50.850441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2f7be501-5400-48d5-afd5-f60b7d95f279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:50.859926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1125/2262] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:18:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:18:50.864457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1d7b4-c18f-4a3e-9181-fb6125fa0c09 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:50.874624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1d7b4-c18f-4a3e-9181-fb6125fa0c09 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:50.879588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:50.880248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:50.880532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:50.880806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:50.881544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:18:50.881913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:50.883548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:50.896077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1138/2263] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:18:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:18:50.900224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7be501-5400-48d5-afd5-f60b7d95f279 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:50.910061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7be501-5400-48d5-afd5-f60b7d95f279 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:50.916096 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346140d-2f33-462c-a53f-c072a835de75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:50.917026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346140d-2f33-462c-a53f-c072a835de75 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:50.917317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346140d-2f33-462c-a53f-c072a835de75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:50.917593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d346140d-2f33-462c-a53f-c072a835de75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:50.918299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346140d-2f33-462c-a53f-c072a835de75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:50.920280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d346140d-2f33-462c-a53f-c072a835de75 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:50.950347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1126/2264] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:18:50 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:18:50.954480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:50.965575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfcf481-c266-4ad5-85cc-ba8d247cc0ff admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:51.027310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521b96c-eafa-4b0c-94a1-295afd434c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:51.028267 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b96c-eafa-4b0c-94a1-295afd434c99 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:51.028738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b96c-eafa-4b0c-94a1-295afd434c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:51.029104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521b96c-eafa-4b0c-94a1-295afd434c99 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:51.029819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521b96c-eafa-4b0c-94a1-295afd434c99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:51.031770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521b96c-eafa-4b0c-94a1-295afd434c99 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:51.052557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1139/2265] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:18:50 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:18:51.056766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346140d-2f33-462c-a53f-c072a835de75 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:51.067544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d346140d-2f33-462c-a53f-c072a835de75 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:51.072694 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad44e2-d3f0-449a-830e-6a80f17d5112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:51.073354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad44e2-d3f0-449a-830e-6a80f17d5112 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:51.073634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad44e2-d3f0-449a-830e-6a80f17d5112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:51.073905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbad44e2-d3f0-449a-830e-6a80f17d5112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:51.074695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad44e2-d3f0-449a-830e-6a80f17d5112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:51.076360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbad44e2-d3f0-449a-830e-6a80f17d5112 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:51.086750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1127/2266] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:18:51.090911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521b96c-eafa-4b0c-94a1-295afd434c99 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:51.105067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521b96c-eafa-4b0c-94a1-295afd434c99 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:51.113725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef23fba-7ad1-49c1-b58b-ce59c66e9702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:51.114663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef23fba-7ad1-49c1-b58b-ce59c66e9702 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:51.115004 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef23fba-7ad1-49c1-b58b-ce59c66e9702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:51.115342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef23fba-7ad1-49c1-b58b-ce59c66e9702 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:51.116434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef23fba-7ad1-49c1-b58b-ce59c66e9702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:51.118041 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef23fba-7ad1-49c1-b58b-ce59c66e9702 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:51.128554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1140/2267] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:18:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:18:51.132773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad44e2-d3f0-449a-830e-6a80f17d5112 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:51.143107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbad44e2-d3f0-449a-830e-6a80f17d5112 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:51.148492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:51.149296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:51.149572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:51.149839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] PATH_INFO: `/v3/projects/74ff8868fb2d476184048c5af5fff5ac/users/e5ea5e44cacf4c258805d3aa3eede5da/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:51.164646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74ff8868fb2d476184048c5af5fff5ac, user_id=e5ea5e44cacf4c258805d3aa3eede5da, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:51.167353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:51.183185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1128/2268] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:18:51 2022] PUT /identity/v3/projects/74ff8868fb2d476184048c5af5fff5ac/users/e5ea5e44cacf4c258805d3aa3eede5da/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:18:51.193429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713d5b-9bda-4571-a6a3-1fead3c1d1c3 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:51.193780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713d5b-9bda-4571-a6a3-1fead3c1d1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:51.194147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10713d5b-9bda-4571-a6a3-1fead3c1d1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:51.226024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-10713d5b-9bda-4571-a6a3-1fead3c1d1c3 None None] MFA Rules not processed for user `e5ea5e44cacf4c258805d3aa3eede5da`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:18:51.227587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-10713d5b-9bda-4571-a6a3-1fead3c1d1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:51.276160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:18:51.301159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1141/2269] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:18:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:18:51.312052 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:51.314084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:51.377044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30775b9a-ad74-47d8-88fc-21e5671b6d85 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:51.380688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:51.381369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:51.381680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:51.381957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:51.390946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:51.392457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:51.402139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1129/2270] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:56.337853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10713d5b-9bda-4571-a6a3-1fead3c1d1c3 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:56.339576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-10713d5b-9bda-4571-a6a3-1fead3c1d1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:56.425350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10713d5b-9bda-4571-a6a3-1fead3c1d1c3 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:56.432028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:56.432679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:56.432959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:56.433228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:56.435031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:56.510638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:56.512406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:56.523328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1142/2271] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:57.177994 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:57.179540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:57.195415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207d22c-d9de-4364-a0b6-8ff409d54157 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:57.200856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccaaa2-36d5-4f8b-9901-4a0e62278c12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:57.201535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccaaa2-36d5-4f8b-9901-4a0e62278c12 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:57.201802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccaaa2-36d5-4f8b-9901-4a0e62278c12 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:57.202080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccaaa2-36d5-4f8b-9901-4a0e62278c12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:57.203955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-00ccaaa2-36d5-4f8b-9901-4a0e62278c12 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:18:57.217345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccaaa2-36d5-4f8b-9901-4a0e62278c12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:57.219067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccaaa2-36d5-4f8b-9901-4a0e62278c12 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:57.230015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1130/2272] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:18:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:18:59.887530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:18:59.952751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67634242-db32-48c7-9041-0c38ed755df2 service nova] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:18:59.957837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:18:59.958695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:18:59.958962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:18:59.959233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] PATH_INFO: `/v3/users/e5ea5e44cacf4c258805d3aa3eede5da` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:18:59.960534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5ea5e44cacf4c258805d3aa3eede5da)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:18:59.962477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:18:59.998511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:00.020077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5ea5e44cacf4c258805d3aa3eede5da'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.028086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5ea5e44cacf4c258805d3aa3eede5da'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.028818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5ea5e44cacf4c258805d3aa3eede5da'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.039626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5ea5e44cacf4c258805d3aa3eede5da'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.040605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5ea5e44cacf4c258805d3aa3eede5da'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.063182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1143/2273] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:18:59 2022] DELETE /identity/v3/users/e5ea5e44cacf4c258805d3aa3eede5da => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:00.068762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa156a8-aca7-497b-8c86-baf1164d4593 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:00.069214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa156a8-aca7-497b-8c86-baf1164d4593 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:00.069347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa156a8-aca7-497b-8c86-baf1164d4593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:00.087355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9aa156a8-aca7-497b-8c86-baf1164d4593 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:00.088545 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9aa156a8-aca7-497b-8c86-baf1164d4593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:00.148322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1131/2274] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:00.158252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:00.159870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:00.220120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe764c0-c541-4ad7-b353-ba2033d0ca31 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:00.224116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db19485-36a1-47d4-9b65-21d4270ecec0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:00.224763 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db19485-36a1-47d4-9b65-21d4270ecec0 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:00.225049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db19485-36a1-47d4-9b65-21d4270ecec0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:00.225323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db19485-36a1-47d4-9b65-21d4270ecec0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:00.235028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db19485-36a1-47d4-9b65-21d4270ecec0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:00.236510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db19485-36a1-47d4-9b65-21d4270ecec0 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:00.246636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1144/2275] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:19:00.527448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa156a8-aca7-497b-8c86-baf1164d4593 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:00.529234 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9aa156a8-aca7-497b-8c86-baf1164d4593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:00.549400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa156a8-aca7-497b-8c86-baf1164d4593 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:00.556349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:00.557382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:00.557727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:00.558079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] PATH_INFO: `/v3/projects/74ff8868fb2d476184048c5af5fff5ac` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:00.559460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] RBAC: Authorizing `identity:delete_project(project_id=74ff8868fb2d476184048c5af5fff5ac)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:00.561563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:00.563367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:00.632236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74ff8868fb2d476184048c5af5fff5ac', 'request_id': 'req-a397be39-8c61-474e-b061-91e896da775e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.640681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74ff8868fb2d476184048c5af5fff5ac', 'request_id': 'req-a397be39-8c61-474e-b061-91e896da775e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.646481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74ff8868fb2d476184048c5af5fff5ac', 'request_id': 'req-a397be39-8c61-474e-b061-91e896da775e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.657950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] The token cache is being invalidate because project 74ff8868fb2d476184048c5af5fff5ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:00.658237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.658897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a397be39-8c61-474e-b061-91e896da775e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:00.660516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1132/2276] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:00 2022] DELETE /identity/v3/projects/74ff8868fb2d476184048c5af5fff5ac => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:00.665685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad43a0-8e0e-46b6-acfd-f1a1153a48c1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:00.665979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad43a0-8e0e-46b6-acfd-f1a1153a48c1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:00.666255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad43a0-8e0e-46b6-acfd-f1a1153a48c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:00.685100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8aad43a0-8e0e-46b6-acfd-f1a1153a48c1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:00.686297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8aad43a0-8e0e-46b6-acfd-f1a1153a48c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:00.745908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1145/2277] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:00.759066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65f8bd-c157-4541-9b5d-384f8866ee69 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:00.759465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65f8bd-c157-4541-9b5d-384f8866ee69 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:00.759809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca65f8bd-c157-4541-9b5d-384f8866ee69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:00.780211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ca65f8bd-c157-4541-9b5d-384f8866ee69 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:00.785182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ca65f8bd-c157-4541-9b5d-384f8866ee69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:00.796973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1133/2278] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:00.801539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad43a0-8e0e-46b6-acfd-f1a1153a48c1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:00.814893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad43a0-8e0e-46b6-acfd-f1a1153a48c1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:00.822557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:00.823214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:00.823522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:00.823792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:00.824529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:19:00.824831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:00.826438 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:00.838935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1146/2279] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:19:00.843393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65f8bd-c157-4541-9b5d-384f8866ee69 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:00.853083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca65f8bd-c157-4541-9b5d-384f8866ee69 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:00.858742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb91b8-30b9-471b-be81-1177c06600d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:00.859426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb91b8-30b9-471b-be81-1177c06600d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:00.859714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb91b8-30b9-471b-be81-1177c06600d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:00.859990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb91b8-30b9-471b-be81-1177c06600d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:00.860687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb91b8-30b9-471b-be81-1177c06600d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:00.862622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb91b8-30b9-471b-be81-1177c06600d1 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:00.894192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1134/2280] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:00 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:00.898829 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:00.908787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19d00a7-a6db-4111-b953-15681b334171 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:00.937779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:00.964918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61d3f2-a0ea-4cd5-bd7c-6cdd24b96d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:00.965594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61d3f2-a0ea-4cd5-bd7c-6cdd24b96d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:00.965875 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61d3f2-a0ea-4cd5-bd7c-6cdd24b96d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:00.966149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61d3f2-a0ea-4cd5-bd7c-6cdd24b96d46 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:00.966838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61d3f2-a0ea-4cd5-bd7c-6cdd24b96d46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:00.968805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61d3f2-a0ea-4cd5-bd7c-6cdd24b96d46 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:00.990803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1147/2281] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:00 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:00.995107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb91b8-30b9-471b-be81-1177c06600d1 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.005653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb91b8-30b9-471b-be81-1177c06600d1 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.010456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342322ce-8d85-483a-b6c9-e283c665c019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.011132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342322ce-8d85-483a-b6c9-e283c665c019 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.011458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342322ce-8d85-483a-b6c9-e283c665c019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.011737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342322ce-8d85-483a-b6c9-e283c665c019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.012477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342322ce-8d85-483a-b6c9-e283c665c019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.014098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342322ce-8d85-483a-b6c9-e283c665c019 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.025437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1135/2282] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:01.029232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61d3f2-a0ea-4cd5-bd7c-6cdd24b96d46 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.039473 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61d3f2-a0ea-4cd5-bd7c-6cdd24b96d46 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.045351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf384e5-c0ee-4b8b-8578-804af1d3cebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.046026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf384e5-c0ee-4b8b-8578-804af1d3cebe admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.046301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf384e5-c0ee-4b8b-8578-804af1d3cebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.046570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf384e5-c0ee-4b8b-8578-804af1d3cebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.047304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf384e5-c0ee-4b8b-8578-804af1d3cebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.048908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf384e5-c0ee-4b8b-8578-804af1d3cebe admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.059361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1148/2283] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:19:01.063739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342322ce-8d85-483a-b6c9-e283c665c019 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.074341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342322ce-8d85-483a-b6c9-e283c665c019 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.080143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36f75d-f6dc-4ecd-9f0b-099da8cf8b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.081032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36f75d-f6dc-4ecd-9f0b-099da8cf8b9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.081316 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36f75d-f6dc-4ecd-9f0b-099da8cf8b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.081595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb36f75d-f6dc-4ecd-9f0b-099da8cf8b9a admin admin] PATH_INFO: `/v3/projects/7716edbdff2b45b3a850f97d873cdcc8/users/b6e78e66e6154e9eb705fd6332c00896/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.097929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36f75d-f6dc-4ecd-9f0b-099da8cf8b9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7716edbdff2b45b3a850f97d873cdcc8, user_id=b6e78e66e6154e9eb705fd6332c00896, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.100662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb36f75d-f6dc-4ecd-9f0b-099da8cf8b9a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.117066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1136/2284] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:01 2022] PUT /identity/v3/projects/7716edbdff2b45b3a850f97d873cdcc8/users/b6e78e66e6154e9eb705fd6332c00896/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:01.127469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96997654-3d12-4228-8b70-965eede56243 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.127760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96997654-3d12-4228-8b70-965eede56243 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.128037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96997654-3d12-4228-8b70-965eede56243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.157018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-96997654-3d12-4228-8b70-965eede56243 None None] MFA Rules not processed for user `b6e78e66e6154e9eb705fd6332c00896`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:01.158231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-96997654-3d12-4228-8b70-965eede56243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:01.223031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1149/2285] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:19:01.227855 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36f75d-f6dc-4ecd-9f0b-099da8cf8b9a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.290546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb36f75d-f6dc-4ecd-9f0b-099da8cf8b9a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.295519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bd500-78a0-49e1-81db-c8aed42dd5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.296182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd500-78a0-49e1-81db-c8aed42dd5ce admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.296469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd500-78a0-49e1-81db-c8aed42dd5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.296736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd500-78a0-49e1-81db-c8aed42dd5ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.297448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bd500-78a0-49e1-81db-c8aed42dd5ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.299394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bd500-78a0-49e1-81db-c8aed42dd5ce admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.330988 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1137/2286] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:01 2022] POST /identity/v3/projects => generated 363 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:01.334847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96997654-3d12-4228-8b70-965eede56243 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.345635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96997654-3d12-4228-8b70-965eede56243 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.405152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022cdb-ecd1-4690-a312-de46f9fddefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.406003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022cdb-ecd1-4690-a312-de46f9fddefb admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.406399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022cdb-ecd1-4690-a312-de46f9fddefb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.406771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03022cdb-ecd1-4690-a312-de46f9fddefb admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.407909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03022cdb-ecd1-4690-a312-de46f9fddefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.410981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03022cdb-ecd1-4690-a312-de46f9fddefb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.435757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1150/2287] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:01 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:01.440362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bd500-78a0-49e1-81db-c8aed42dd5ce admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.452809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bd500-78a0-49e1-81db-c8aed42dd5ce admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.457822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51975adb-abc2-40b8-aaff-41e7bbcdc9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.458475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51975adb-abc2-40b8-aaff-41e7bbcdc9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.458754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51975adb-abc2-40b8-aaff-41e7bbcdc9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.459023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51975adb-abc2-40b8-aaff-41e7bbcdc9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.459871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51975adb-abc2-40b8-aaff-41e7bbcdc9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.461693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51975adb-abc2-40b8-aaff-41e7bbcdc9e9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.467504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:01.473642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1138/2288] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:01.478127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022cdb-ecd1-4690-a312-de46f9fddefb admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.488363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03022cdb-ecd1-4690-a312-de46f9fddefb admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.493550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb874-1f4d-4147-b5dd-83410eb6c991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.494456 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb874-1f4d-4147-b5dd-83410eb6c991 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.494735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb874-1f4d-4147-b5dd-83410eb6c991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.495007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bb874-1f4d-4147-b5dd-83410eb6c991 admin admin] PATH_INFO: `/v3/projects/0510f36e71d845a9b88d3556da294f8d/users/9ebaf323110940008eacd1e372224485/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.509976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bb874-1f4d-4147-b5dd-83410eb6c991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0510f36e71d845a9b88d3556da294f8d, user_id=9ebaf323110940008eacd1e372224485, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.512663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bb874-1f4d-4147-b5dd-83410eb6c991 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.528044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1151/2289] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:01 2022] PUT /identity/v3/projects/0510f36e71d845a9b88d3556da294f8d/users/9ebaf323110940008eacd1e372224485/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:01.531987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51975adb-abc2-40b8-aaff-41e7bbcdc9e9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.542262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51975adb-abc2-40b8-aaff-41e7bbcdc9e9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.607055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8499b5-439e-4c5f-96ab-0c329f8f59c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.607755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8499b5-439e-4c5f-96ab-0c329f8f59c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.608044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8499b5-439e-4c5f-96ab-0c329f8f59c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.608319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8499b5-439e-4c5f-96ab-0c329f8f59c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.609055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8499b5-439e-4c5f-96ab-0c329f8f59c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.610670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8499b5-439e-4c5f-96ab-0c329f8f59c4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.621113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1139/2290] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:01.625786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb874-1f4d-4147-b5dd-83410eb6c991 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.638750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bb874-1f4d-4147-b5dd-83410eb6c991 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.645177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.646065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.646350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.646651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] PATH_INFO: `/v3/projects/0510f36e71d845a9b88d3556da294f8d/users/9ebaf323110940008eacd1e372224485/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.648828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0510f36e71d845a9b88d3556da294f8d, user_id=9ebaf323110940008eacd1e372224485, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.651525 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.653840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:01.672099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1152/2291] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:01 2022] PUT /identity/v3/projects/0510f36e71d845a9b88d3556da294f8d/users/9ebaf323110940008eacd1e372224485/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:01.674867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f5ac1-6b81-4582-bb8d-47383ad4a45c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.675158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f5ac1-6b81-4582-bb8d-47383ad4a45c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.675561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14f5ac1-6b81-4582-bb8d-47383ad4a45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.702967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b14f5ac1-6b81-4582-bb8d-47383ad4a45c None None] MFA Rules not processed for user `9ebaf323110940008eacd1e372224485`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:01.704225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b14f5ac1-6b81-4582-bb8d-47383ad4a45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:01.780440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1140/2292] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:01 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:01.792132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.794109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:01.850487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4c1dd2-7d00-4ab0-a3ab-a6abe05f8dd5 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.854468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695bcd1-2751-4f0a-9281-57fa1a3b6f20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.855105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695bcd1-2751-4f0a-9281-57fa1a3b6f20 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.855414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695bcd1-2751-4f0a-9281-57fa1a3b6f20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.855690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695bcd1-2751-4f0a-9281-57fa1a3b6f20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:01.864786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695bcd1-2751-4f0a-9281-57fa1a3b6f20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:01.866216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695bcd1-2751-4f0a-9281-57fa1a3b6f20 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:01.875450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1153/2293] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:01.950733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f5ac1-6b81-4582-bb8d-47383ad4a45c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:01.960976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14f5ac1-6b81-4582-bb8d-47383ad4a45c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:01.965414 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6482c-fed0-46ee-87be-2a5ffa5ee9e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:01.966062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6482c-fed0-46ee-87be-2a5ffa5ee9e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:01.966345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6482c-fed0-46ee-87be-2a5ffa5ee9e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:01.966614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6482c-fed0-46ee-87be-2a5ffa5ee9e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:02.040370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6482c-fed0-46ee-87be-2a5ffa5ee9e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:02.041863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6482c-fed0-46ee-87be-2a5ffa5ee9e4 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:02.050987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1141/2294] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:11.266901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695bcd1-2751-4f0a-9281-57fa1a3b6f20 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:11.340270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695bcd1-2751-4f0a-9281-57fa1a3b6f20 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:11.346206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:11.347083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:11.347388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:11.347637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] PATH_INFO: `/v3/users/b6e78e66e6154e9eb705fd6332c00896` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:11.349315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6e78e66e6154e9eb705fd6332c00896)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:11.352403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:11.412637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6e78e66e6154e9eb705fd6332c00896'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:11.424260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6e78e66e6154e9eb705fd6332c00896'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:11.425385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6e78e66e6154e9eb705fd6332c00896'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:11.437420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6e78e66e6154e9eb705fd6332c00896'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:11.438146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6e78e66e6154e9eb705fd6332c00896'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:11.455295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1154/2295] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:11 2022] DELETE /identity/v3/users/b6e78e66e6154e9eb705fd6332c00896 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:11.460045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c886-af41-4186-ad50-3aed1f017fef None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:11.460543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c886-af41-4186-ad50-3aed1f017fef None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:11.460642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72c886-af41-4186-ad50-3aed1f017fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:11.478833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-7c72c886-af41-4186-ad50-3aed1f017fef None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:11.480064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7c72c886-af41-4186-ad50-3aed1f017fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:11.492667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:11.544607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1142/2296] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:11.556330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:11.558258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:11.577227 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:11.617434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a086471f-e177-4064-b995-87d698b21449 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:11.621633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1c7bf-6b67-4888-aa11-f82a6ccd3fed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:11.622277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1c7bf-6b67-4888-aa11-f82a6ccd3fed service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:11.622570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1c7bf-6b67-4888-aa11-f82a6ccd3fed service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:11.622915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1c7bf-6b67-4888-aa11-f82a6ccd3fed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:11.632158 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1c7bf-6b67-4888-aa11-f82a6ccd3fed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:11.633601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1c7bf-6b67-4888-aa11-f82a6ccd3fed service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:11.643288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1155/2297] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:11.913497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72c886-af41-4186-ad50-3aed1f017fef None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:11.915071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7c72c886-af41-4186-ad50-3aed1f017fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:11.927564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72c886-af41-4186-ad50-3aed1f017fef None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:11.932757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:11.933629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:11.933911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:11.934182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] PATH_INFO: `/v3/projects/7716edbdff2b45b3a850f97d873cdcc8` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:11.935402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] RBAC: Authorizing `identity:delete_project(project_id=7716edbdff2b45b3a850f97d873cdcc8)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:11.937345 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:11.988193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7716edbdff2b45b3a850f97d873cdcc8', 'request_id': 'req-0513134e-aebd-4521-8131-1509dfaadbba'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:11.997689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7716edbdff2b45b3a850f97d873cdcc8', 'request_id': 'req-0513134e-aebd-4521-8131-1509dfaadbba'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.003003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7716edbdff2b45b3a850f97d873cdcc8', 'request_id': 'req-0513134e-aebd-4521-8131-1509dfaadbba'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.013941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] The token cache is being invalidate because project 7716edbdff2b45b3a850f97d873cdcc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:12.014224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.014880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.016431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1143/2298] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:11 2022] DELETE /identity/v3/projects/7716edbdff2b45b3a850f97d873cdcc8 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:19:12.021134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1c7bf-6b67-4888-aa11-f82a6ccd3fed service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:12.031652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1c7bf-6b67-4888-aa11-f82a6ccd3fed service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:12.090505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:12.091421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:12.091701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:12.091972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] PATH_INFO: `/v3/users/9ebaf323110940008eacd1e372224485` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:12.093157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ebaf323110940008eacd1e372224485)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:12.095078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:12.158287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ebaf323110940008eacd1e372224485'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.165470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ebaf323110940008eacd1e372224485'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.166200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ebaf323110940008eacd1e372224485'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.174164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ebaf323110940008eacd1e372224485'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.174824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-556d2f0e-83cb-489c-9796-4e50f4811381 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ebaf323110940008eacd1e372224485'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.192293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1156/2299] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:12 2022] DELETE /identity/v3/users/9ebaf323110940008eacd1e372224485 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:12.467846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:12.469267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:12.493440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513134e-aebd-4521-8131-1509dfaadbba admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:12.549094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:12.549962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:12.550243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:12.550512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] PATH_INFO: `/v3/projects/0510f36e71d845a9b88d3556da294f8d` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:12.551773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0510f36e71d845a9b88d3556da294f8d)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:12.553688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:12.597510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:12.600280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0510f36e71d845a9b88d3556da294f8d', 'request_id': 'req-f0cd2190-6333-4492-a9b7-a656a036bf98'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.610030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0510f36e71d845a9b88d3556da294f8d', 'request_id': 'req-f0cd2190-6333-4492-a9b7-a656a036bf98'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.616009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0510f36e71d845a9b88d3556da294f8d', 'request_id': 'req-f0cd2190-6333-4492-a9b7-a656a036bf98'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.627172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] The token cache is being invalidate because project 0510f36e71d845a9b88d3556da294f8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:12.627521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.628290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f0cd2190-6333-4492-a9b7-a656a036bf98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:12.629998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1144/2300] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:12 2022] DELETE /identity/v3/projects/0510f36e71d845a9b88d3556da294f8d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:12.635062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d028-1407-4b1e-be50-eed9c125d90e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:12.635593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d028-1407-4b1e-be50-eed9c125d90e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:12.635720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b025d028-1407-4b1e-be50-eed9c125d90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:12.654571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b025d028-1407-4b1e-be50-eed9c125d90e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:12.655841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b025d028-1407-4b1e-be50-eed9c125d90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:12.691306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:12.720445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1157/2301] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:12.728401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718358-309a-448f-9777-9f5a73b504ea None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:12.728689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718358-309a-448f-9777-9f5a73b504ea None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:12.728976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee718358-309a-448f-9777-9f5a73b504ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:12.748020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ee718358-309a-448f-9777-9f5a73b504ea None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:12.749236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ee718358-309a-448f-9777-9f5a73b504ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:12.757704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1145/2302] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:12.761979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025d028-1407-4b1e-be50-eed9c125d90e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:12.772304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b025d028-1407-4b1e-be50-eed9c125d90e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:12.780334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:12.781023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:12.781299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:12.781563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:12.782298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:19:12.782600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:12.784241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:12.796594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1158/2303] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:19:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:19:12.801073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee718358-309a-448f-9777-9f5a73b504ea None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:12.810862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee718358-309a-448f-9777-9f5a73b504ea None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:12.816521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94850589-d460-43da-8475-6861df40bfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:12.817170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94850589-d460-43da-8475-6861df40bfce admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:12.817452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94850589-d460-43da-8475-6861df40bfce admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:12.817729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94850589-d460-43da-8475-6861df40bfce admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:12.818428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94850589-d460-43da-8475-6861df40bfce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:12.820416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94850589-d460-43da-8475-6861df40bfce admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:12.848761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1146/2304] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:12 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:12.852902 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:12.862898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96048f5-acb8-4d8c-b245-6197cd32e8b2 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:12.918951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25938470-a002-4aef-97ba-c36c9bc62536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:12.919638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25938470-a002-4aef-97ba-c36c9bc62536 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:12.919916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25938470-a002-4aef-97ba-c36c9bc62536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:12.920177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25938470-a002-4aef-97ba-c36c9bc62536 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:12.920865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25938470-a002-4aef-97ba-c36c9bc62536 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:12.922770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25938470-a002-4aef-97ba-c36c9bc62536 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:12.945825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1159/2305] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:12 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:12.950212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94850589-d460-43da-8475-6861df40bfce admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:12.960581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94850589-d460-43da-8475-6861df40bfce admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:12.965934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d0491-e15f-444b-b875-c08815300bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:12.966595 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d0491-e15f-444b-b875-c08815300bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:12.966877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d0491-e15f-444b-b875-c08815300bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:12.967149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d0491-e15f-444b-b875-c08815300bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:12.967949 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d0491-e15f-444b-b875-c08815300bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:12.969562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d0491-e15f-444b-b875-c08815300bc8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:12.980424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1147/2306] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:12.984464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25938470-a002-4aef-97ba-c36c9bc62536 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:12.994604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25938470-a002-4aef-97ba-c36c9bc62536 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:12.999967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1ab3c-33e6-4f22-a603-b66a6a4989be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.000747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1ab3c-33e6-4f22-a603-b66a6a4989be admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.001031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1ab3c-33e6-4f22-a603-b66a6a4989be admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.001307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1ab3c-33e6-4f22-a603-b66a6a4989be admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.002042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1ab3c-33e6-4f22-a603-b66a6a4989be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.003674 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1ab3c-33e6-4f22-a603-b66a6a4989be admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.014535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1160/2307] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:13.019064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d0491-e15f-444b-b875-c08815300bc8 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.029905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d0491-e15f-444b-b875-c08815300bc8 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.034881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961553fe-3162-44a7-998b-54f0677086de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.035815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961553fe-3162-44a7-998b-54f0677086de admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.036100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961553fe-3162-44a7-998b-54f0677086de admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.036374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961553fe-3162-44a7-998b-54f0677086de admin admin] PATH_INFO: `/v3/projects/c3a589abcddb46a5ab5ab79bdfe42946/users/39394168091d48c0b4449419144fdb64/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.051709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961553fe-3162-44a7-998b-54f0677086de admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a589abcddb46a5ab5ab79bdfe42946, user_id=39394168091d48c0b4449419144fdb64, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.054401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961553fe-3162-44a7-998b-54f0677086de admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.069329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1148/2308] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:13 2022] PUT /identity/v3/projects/c3a589abcddb46a5ab5ab79bdfe42946/users/39394168091d48c0b4449419144fdb64/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:13.079892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80cc0-a8e0-4baf-a1ad-87697593a1fa None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.080186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80cc0-a8e0-4baf-a1ad-87697593a1fa None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.080462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d80cc0-a8e0-4baf-a1ad-87697593a1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.106643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-82d80cc0-a8e0-4baf-a1ad-87697593a1fa None None] MFA Rules not processed for user `39394168091d48c0b4449419144fdb64`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:13.107907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-82d80cc0-a8e0-4baf-a1ad-87697593a1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:13.174657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1161/2309] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:13.179453 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961553fe-3162-44a7-998b-54f0677086de admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.243892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961553fe-3162-44a7-998b-54f0677086de admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.249067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6306774-9d15-41cf-bf28-774543b974ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.249727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6306774-9d15-41cf-bf28-774543b974ae admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.250043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6306774-9d15-41cf-bf28-774543b974ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.250315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6306774-9d15-41cf-bf28-774543b974ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.251013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6306774-9d15-41cf-bf28-774543b974ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.253059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6306774-9d15-41cf-bf28-774543b974ae admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.282114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1149/2310] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:13 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:13.286479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d80cc0-a8e0-4baf-a1ad-87697593a1fa None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.297537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d80cc0-a8e0-4baf-a1ad-87697593a1fa None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.326789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:13.356483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162e2f9-dd61-4dae-8564-4e16fd34dcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.357156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162e2f9-dd61-4dae-8564-4e16fd34dcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.357428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162e2f9-dd61-4dae-8564-4e16fd34dcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.357697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162e2f9-dd61-4dae-8564-4e16fd34dcd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.358382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162e2f9-dd61-4dae-8564-4e16fd34dcd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.360347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162e2f9-dd61-4dae-8564-4e16fd34dcd0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.380182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1162/2311] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:13 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:13.384170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6306774-9d15-41cf-bf28-774543b974ae admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.394970 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6306774-9d15-41cf-bf28-774543b974ae admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.399783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548681b-ba68-4551-bc8a-cffcc8689807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.400440 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548681b-ba68-4551-bc8a-cffcc8689807 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.400719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548681b-ba68-4551-bc8a-cffcc8689807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.401016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0548681b-ba68-4551-bc8a-cffcc8689807 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.401747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548681b-ba68-4551-bc8a-cffcc8689807 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.403373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0548681b-ba68-4551-bc8a-cffcc8689807 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.413978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1150/2312] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:13.417814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162e2f9-dd61-4dae-8564-4e16fd34dcd0 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.428173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162e2f9-dd61-4dae-8564-4e16fd34dcd0 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.433293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc1ebbf-8b44-42da-8a59-70c5a12edf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.433992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1ebbf-8b44-42da-8a59-70c5a12edf01 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.434280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1ebbf-8b44-42da-8a59-70c5a12edf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.434551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc1ebbf-8b44-42da-8a59-70c5a12edf01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.435304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1ebbf-8b44-42da-8a59-70c5a12edf01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.436939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc1ebbf-8b44-42da-8a59-70c5a12edf01 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.447870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1163/2313] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:13.452530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548681b-ba68-4551-bc8a-cffcc8689807 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.463411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0548681b-ba68-4551-bc8a-cffcc8689807 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.469578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b9672-c7f2-4105-a39f-f5f7b6a128ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.470487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9672-c7f2-4105-a39f-f5f7b6a128ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.470756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9672-c7f2-4105-a39f-f5f7b6a128ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.471036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b9672-c7f2-4105-a39f-f5f7b6a128ef admin admin] PATH_INFO: `/v3/projects/c8594c85552c4cae93292666a28e8447/users/cbc79649b06f460989f54a189987464c/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.486648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b9672-c7f2-4105-a39f-f5f7b6a128ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8594c85552c4cae93292666a28e8447, user_id=cbc79649b06f460989f54a189987464c, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.489414 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b9672-c7f2-4105-a39f-f5f7b6a128ef admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.504078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1151/2314] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:13 2022] PUT /identity/v3/projects/c8594c85552c4cae93292666a28e8447/users/cbc79649b06f460989f54a189987464c/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:13.517308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceef54-2a08-4cc8-b8ca-0dbafa460e6e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.517661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceef54-2a08-4cc8-b8ca-0dbafa460e6e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.518286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edceef54-2a08-4cc8-b8ca-0dbafa460e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.547238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-edceef54-2a08-4cc8-b8ca-0dbafa460e6e None None] MFA Rules not processed for user `cbc79649b06f460989f54a189987464c`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:13.548530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-edceef54-2a08-4cc8-b8ca-0dbafa460e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:13.611733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1164/2315] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:19:13.616130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b9672-c7f2-4105-a39f-f5f7b6a128ef admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.637327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:13.679075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b9672-c7f2-4105-a39f-f5f7b6a128ef admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.684134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146f7c-0a77-4c36-b63a-9915e0a1e920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.684778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146f7c-0a77-4c36-b63a-9915e0a1e920 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.685059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146f7c-0a77-4c36-b63a-9915e0a1e920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.685325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146f7c-0a77-4c36-b63a-9915e0a1e920 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.686023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a146f7c-0a77-4c36-b63a-9915e0a1e920 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.687976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a146f7c-0a77-4c36-b63a-9915e0a1e920 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.717013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1152/2316] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:13 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:19:13.721210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edceef54-2a08-4cc8-b8ca-0dbafa460e6e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.733687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edceef54-2a08-4cc8-b8ca-0dbafa460e6e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.804321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141816a2-d374-41e8-9da8-75e7cc3dc85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.805006 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141816a2-d374-41e8-9da8-75e7cc3dc85e admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.805295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141816a2-d374-41e8-9da8-75e7cc3dc85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.805566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141816a2-d374-41e8-9da8-75e7cc3dc85e admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.806270 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141816a2-d374-41e8-9da8-75e7cc3dc85e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.808354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141816a2-d374-41e8-9da8-75e7cc3dc85e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.832475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1165/2317] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:13 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:13.836554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146f7c-0a77-4c36-b63a-9915e0a1e920 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.847086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146f7c-0a77-4c36-b63a-9915e0a1e920 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.852814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b1265-a0ba-4868-ae78-ab314f7df2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.853985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1265-a0ba-4868-ae78-ab314f7df2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.854272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1265-a0ba-4868-ae78-ab314f7df2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.854547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29b1265-a0ba-4868-ae78-ab314f7df2f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.855301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b1265-a0ba-4868-ae78-ab314f7df2f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.856938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29b1265-a0ba-4868-ae78-ab314f7df2f5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.867755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1153/2318] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:13.872132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141816a2-d374-41e8-9da8-75e7cc3dc85e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.882078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141816a2-d374-41e8-9da8-75e7cc3dc85e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.886945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d212a-5f28-47b0-bf55-4e50400d8b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.887844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d212a-5f28-47b0-bf55-4e50400d8b7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.888125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d212a-5f28-47b0-bf55-4e50400d8b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.888372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451d212a-5f28-47b0-bf55-4e50400d8b7c admin admin] PATH_INFO: `/v3/projects/7a580e6f5d3c4573ba2b3b3170fef9fd/users/8437f3e9453a46d39ec20c77d85c008c/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.904820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d212a-5f28-47b0-bf55-4e50400d8b7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a580e6f5d3c4573ba2b3b3170fef9fd, user_id=8437f3e9453a46d39ec20c77d85c008c, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.907490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451d212a-5f28-47b0-bf55-4e50400d8b7c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:13.922729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1166/2319] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:13 2022] PUT /identity/v3/projects/7a580e6f5d3c4573ba2b3b3170fef9fd/users/8437f3e9453a46d39ec20c77d85c008c/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:13.926839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b1265-a0ba-4868-ae78-ab314f7df2f5 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:13.937015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29b1265-a0ba-4868-ae78-ab314f7df2f5 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:13.994642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cf7a79-8aa7-4262-94e4-46eb935e682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:13.995334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf7a79-8aa7-4262-94e4-46eb935e682b admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:13.995619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf7a79-8aa7-4262-94e4-46eb935e682b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:13.995895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cf7a79-8aa7-4262-94e4-46eb935e682b admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:13.996632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf7a79-8aa7-4262-94e4-46eb935e682b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:13.998228 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cf7a79-8aa7-4262-94e4-46eb935e682b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:14.008078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1154/2320] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:14.012301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d212a-5f28-47b0-bf55-4e50400d8b7c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:14.023484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451d212a-5f28-47b0-bf55-4e50400d8b7c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:14.028602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:14.029558 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:14.029908 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:14.030192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] PATH_INFO: `/v3/projects/7a580e6f5d3c4573ba2b3b3170fef9fd/users/8437f3e9453a46d39ec20c77d85c008c/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:14.032255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a580e6f5d3c4573ba2b3b3170fef9fd, user_id=8437f3e9453a46d39ec20c77d85c008c, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:14.034923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:14.037206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:14.048311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1167/2321] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:14 2022] PUT /identity/v3/projects/7a580e6f5d3c4573ba2b3b3170fef9fd/users/8437f3e9453a46d39ec20c77d85c008c/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:14.062463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd594031-6d3f-4f1b-926d-5658154c9f4c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:14.062767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd594031-6d3f-4f1b-926d-5658154c9f4c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:14.063040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd594031-6d3f-4f1b-926d-5658154c9f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:14.091782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-bd594031-6d3f-4f1b-926d-5658154c9f4c None None] MFA Rules not processed for user `8437f3e9453a46d39ec20c77d85c008c`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:14.093029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bd594031-6d3f-4f1b-926d-5658154c9f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:14.178029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1155/2322] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:14.190166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:14.192477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:14.255189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aca69a-8f83-4172-b358-e9f83feeda07 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:14.259115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a03dfe-7e61-4f96-ba55-a7101c0ad54f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:14.259783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a03dfe-7e61-4f96-ba55-a7101c0ad54f service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:14.260064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a03dfe-7e61-4f96-ba55-a7101c0ad54f service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:14.260333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a03dfe-7e61-4f96-ba55-a7101c0ad54f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:14.270416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a03dfe-7e61-4f96-ba55-a7101c0ad54f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:14.271885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a03dfe-7e61-4f96-ba55-a7101c0ad54f service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:14.281239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1168/2323] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:14.804591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd594031-6d3f-4f1b-926d-5658154c9f4c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:14.806079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bd594031-6d3f-4f1b-926d-5658154c9f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:14.818916 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd594031-6d3f-4f1b-926d-5658154c9f4c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:14.823895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:14.824889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:14.825225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:14.825578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:14.840538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:14.889171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:14.890633 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:14.900582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1156/2324] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:18.096890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6486c-06ef-4d06-a265-08559ffd1fc6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:18.097383 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6486c-06ef-4d06-a265-08559ffd1fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:18.097503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca6486c-06ef-4d06-a265-08559ffd1fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:18.115561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8ca6486c-06ef-4d06-a265-08559ffd1fc6 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:18.116759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8ca6486c-06ef-4d06-a265-08559ffd1fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:18.182536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1169/2325] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:19:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:18.234810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:18.236449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:18.310243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17afc5-c996-4a91-b645-8dd0384efeaf service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:18.315471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:18.316124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:18.316404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:18.316679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:18.324826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:18.326487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:18.334651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1157/2326] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:19:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:19:18.980131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6486c-06ef-4d06-a265-08559ffd1fc6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:18.993152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca6486c-06ef-4d06-a265-08559ffd1fc6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:18.997975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4800f52-3404-4cf0-98f7-f722499436c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:18.998839 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4800f52-3404-4cf0-98f7-f722499436c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:18.999223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4800f52-3404-4cf0-98f7-f722499436c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:18.999631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4800f52-3404-4cf0-98f7-f722499436c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:19.070137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4800f52-3404-4cf0-98f7-f722499436c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:19.071844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4800f52-3404-4cf0-98f7-f722499436c5 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:19.082665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1170/2327] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:19.632787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:19.634351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:19.792537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc70fcca-0daa-40e2-88b4-8da812ae6489 service swift] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:19.803077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4746a88-5937-431e-a2b0-a0dfaf26dc43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:19.803881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746a88-5937-431e-a2b0-a0dfaf26dc43 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:19.804162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746a88-5937-431e-a2b0-a0dfaf26dc43 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:19.804435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746a88-5937-431e-a2b0-a0dfaf26dc43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:19.808724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e4746a88-5937-431e-a2b0-a0dfaf26dc43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:19.833104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4746a88-5937-431e-a2b0-a0dfaf26dc43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:19.835119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4746a88-5937-431e-a2b0-a0dfaf26dc43 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:19.858665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1158/2328] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:21.521824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4800f52-3404-4cf0-98f7-f722499436c5 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:21.575866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:21.605529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4800f52-3404-4cf0-98f7-f722499436c5 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:21.611154 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:21.612068 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:21.612348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:21.612623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] PATH_INFO: `/v3/users/39394168091d48c0b4449419144fdb64` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:21.614554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] RBAC: Authorizing `identity:delete_user(user_id=39394168091d48c0b4449419144fdb64)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:21.616616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:21.679430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39394168091d48c0b4449419144fdb64'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:21.687967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39394168091d48c0b4449419144fdb64'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:21.688658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39394168091d48c0b4449419144fdb64'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:21.700483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39394168091d48c0b4449419144fdb64'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:21.701414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39394168091d48c0b4449419144fdb64'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:21.729109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1171/2329] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:21 2022] DELETE /identity/v3/users/39394168091d48c0b4449419144fdb64 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:19:21.730666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1550b-1f63-446b-a309-c4cc2aec00e7 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:21.731138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1550b-1f63-446b-a309-c4cc2aec00e7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:21.731235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1550b-1f63-446b-a309-c4cc2aec00e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:21.758536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-84a1550b-1f63-446b-a309-c4cc2aec00e7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:21.759950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-84a1550b-1f63-446b-a309-c4cc2aec00e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:21.816310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:21.866154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1159/2330] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:21.876637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:21.878199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:21.962563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5b665-c802-42f2-922c-bd43ad01075b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:21.966136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe19c90-7121-47b1-93fa-46bafb72440c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:21.966785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe19c90-7121-47b1-93fa-46bafb72440c service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:21.967113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe19c90-7121-47b1-93fa-46bafb72440c service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:21.967445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe19c90-7121-47b1-93fa-46bafb72440c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:21.980309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe19c90-7121-47b1-93fa-46bafb72440c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:21.981756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe19c90-7121-47b1-93fa-46bafb72440c service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:21.993070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1172/2331] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:22.300975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1550b-1f63-446b-a309-c4cc2aec00e7 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:22.302237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-84a1550b-1f63-446b-a309-c4cc2aec00e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:22.314822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1550b-1f63-446b-a309-c4cc2aec00e7 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:22.321989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:22.323296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:22.323811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:22.324294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] PATH_INFO: `/v3/projects/c3a589abcddb46a5ab5ab79bdfe42946` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:22.325735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a589abcddb46a5ab5ab79bdfe42946)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:22.331216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:22.408996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a589abcddb46a5ab5ab79bdfe42946', 'request_id': 'req-21f31a98-29bf-4a92-b8ee-f52a9b36789b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.419891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a589abcddb46a5ab5ab79bdfe42946', 'request_id': 'req-21f31a98-29bf-4a92-b8ee-f52a9b36789b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.430802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a589abcddb46a5ab5ab79bdfe42946', 'request_id': 'req-21f31a98-29bf-4a92-b8ee-f52a9b36789b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.444224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] The token cache is being invalidate because project c3a589abcddb46a5ab5ab79bdfe42946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:22.444514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.445184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.450463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1160/2332] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:22 2022] DELETE /identity/v3/projects/c3a589abcddb46a5ab5ab79bdfe42946 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:22.451877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe19c90-7121-47b1-93fa-46bafb72440c service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:22.462883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe19c90-7121-47b1-93fa-46bafb72440c service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:22.532372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:22.533268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:22.533555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:22.533828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] PATH_INFO: `/v3/users/cbc79649b06f460989f54a189987464c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:22.535712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbc79649b06f460989f54a189987464c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:22.537862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:22.545255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:22.614193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbc79649b06f460989f54a189987464c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.627423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbc79649b06f460989f54a189987464c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.628346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbc79649b06f460989f54a189987464c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.637786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbc79649b06f460989f54a189987464c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.638606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbc79649b06f460989f54a189987464c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:22.658126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1173/2333] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:22 2022] DELETE /identity/v3/users/cbc79649b06f460989f54a189987464c => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:22.986546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:22.988132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:23.002756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f31a98-29bf-4a92-b8ee-f52a9b36789b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:23.077497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:23.078996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:23.079554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:23.080068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] PATH_INFO: `/v3/projects/c8594c85552c4cae93292666a28e8447` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:23.081733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8594c85552c4cae93292666a28e8447)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:23.084342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:23.154409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8594c85552c4cae93292666a28e8447', 'request_id': 'req-43023b8c-7166-4efe-a75b-7913f2f172ce'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.158888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:23.173454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8594c85552c4cae93292666a28e8447', 'request_id': 'req-43023b8c-7166-4efe-a75b-7913f2f172ce'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.179489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8594c85552c4cae93292666a28e8447', 'request_id': 'req-43023b8c-7166-4efe-a75b-7913f2f172ce'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.192023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] The token cache is being invalidate because project c8594c85552c4cae93292666a28e8447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:23.192981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.193718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.198495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1161/2334] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:22 2022] DELETE /identity/v3/projects/c8594c85552c4cae93292666a28e8447 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:23.199757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:23.201201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:23.216793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944faa3f-a956-4b01-a6bc-a03f604730c5 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:23.285877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:23.286887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:23.287174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:23.287477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] PATH_INFO: `/v3/users/8437f3e9453a46d39ec20c77d85c008c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:23.292043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] RBAC: Authorizing `identity:delete_user(user_id=8437f3e9453a46d39ec20c77d85c008c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:23.294211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:23.351356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8437f3e9453a46d39ec20c77d85c008c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.360417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8437f3e9453a46d39ec20c77d85c008c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.361143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8437f3e9453a46d39ec20c77d85c008c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.371801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8437f3e9453a46d39ec20c77d85c008c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.372644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-aefa5bfe-7bb3-4d53-aace-aa92eb1e630f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8437f3e9453a46d39ec20c77d85c008c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.389104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:23.394165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1174/2335] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:23 2022] DELETE /identity/v3/users/8437f3e9453a46d39ec20c77d85c008c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:23.713673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:23.715291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:23.730183 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43023b8c-7166-4efe-a75b-7913f2f172ce admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:23.806373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:23.807402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:23.807751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:23.808095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] PATH_INFO: `/v3/projects/7a580e6f5d3c4573ba2b3b3170fef9fd` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:23.809431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a580e6f5d3c4573ba2b3b3170fef9fd)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:23.811636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:23.873053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a580e6f5d3c4573ba2b3b3170fef9fd', 'request_id': 'req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.910451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a580e6f5d3c4573ba2b3b3170fef9fd', 'request_id': 'req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.916687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a580e6f5d3c4573ba2b3b3170fef9fd', 'request_id': 'req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.927834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] The token cache is being invalidate because project 7a580e6f5d3c4573ba2b3b3170fef9fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:23.928119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.928803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-aa8fe4c7-b1b4-49c3-b5be-406f819ba7c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:23.930366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1162/2336] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:23 2022] DELETE /identity/v3/projects/7a580e6f5d3c4573ba2b3b3170fef9fd => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:23.936443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8dcee-83ef-4e76-9661-776ddca6f3d1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:23.936909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8dcee-83ef-4e76-9661-776ddca6f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:23.937017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a8dcee-83ef-4e76-9661-776ddca6f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:23.955557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-76a8dcee-83ef-4e76-9661-776ddca6f3d1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:23.956739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-76a8dcee-83ef-4e76-9661-776ddca6f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:24.027735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1175/2337] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:24.035817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25853d75-8b57-48ea-b5fe-af2af2af06b2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.036290 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25853d75-8b57-48ea-b5fe-af2af2af06b2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.036383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25853d75-8b57-48ea-b5fe-af2af2af06b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.058717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-25853d75-8b57-48ea-b5fe-af2af2af06b2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:24.059997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-25853d75-8b57-48ea-b5fe-af2af2af06b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:24.071092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1163/2338] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:19:24.075345 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8dcee-83ef-4e76-9661-776ddca6f3d1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.086667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a8dcee-83ef-4e76-9661-776ddca6f3d1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.091355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.092072 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.092351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.092623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.093354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:19:24.093653 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.095477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.111479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1176/2339] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:19:24.113158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25853d75-8b57-48ea-b5fe-af2af2af06b2 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.123093 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25853d75-8b57-48ea-b5fe-af2af2af06b2 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.128266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f7a13-178a-42dc-a981-e7cf0ca345e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.128939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f7a13-178a-42dc-a981-e7cf0ca345e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.129220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f7a13-178a-42dc-a981-e7cf0ca345e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.129499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f7a13-178a-42dc-a981-e7cf0ca345e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.130207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f7a13-178a-42dc-a981-e7cf0ca345e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.132194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f7a13-178a-42dc-a981-e7cf0ca345e6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.164748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1164/2340] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:24 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:19:24.166217 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.178172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cf904d-e0c7-4df3-942e-1b5c2daf73a4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.268326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c61510-6859-4e3b-a09e-7bb83306f79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.269177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c61510-6859-4e3b-a09e-7bb83306f79a admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.269467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c61510-6859-4e3b-a09e-7bb83306f79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.269743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c61510-6859-4e3b-a09e-7bb83306f79a admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.273134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c61510-6859-4e3b-a09e-7bb83306f79a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.273413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c61510-6859-4e3b-a09e-7bb83306f79a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.302066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1177/2341] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:24 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:24.303226 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f7a13-178a-42dc-a981-e7cf0ca345e6 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.314682 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f7a13-178a-42dc-a981-e7cf0ca345e6 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.320255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74da29-2d80-4517-90a0-e52d0f124e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.320944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74da29-2d80-4517-90a0-e52d0f124e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.321404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74da29-2d80-4517-90a0-e52d0f124e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.321687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74da29-2d80-4517-90a0-e52d0f124e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.322433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74da29-2d80-4517-90a0-e52d0f124e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.324212 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74da29-2d80-4517-90a0-e52d0f124e24 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.346377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1165/2342] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:24.347616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c61510-6859-4e3b-a09e-7bb83306f79a admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.362968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c61510-6859-4e3b-a09e-7bb83306f79a admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.367649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94d9248-d732-4c4c-a0e3-9f3f1864c388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.368331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d9248-d732-4c4c-a0e3-9f3f1864c388 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.368605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d9248-d732-4c4c-a0e3-9f3f1864c388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.368880 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94d9248-d732-4c4c-a0e3-9f3f1864c388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.369626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d9248-d732-4c4c-a0e3-9f3f1864c388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.371297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94d9248-d732-4c4c-a0e3-9f3f1864c388 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.385358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1178/2343] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:24.386939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74da29-2d80-4517-90a0-e52d0f124e24 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.405053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74da29-2d80-4517-90a0-e52d0f124e24 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.422541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0f977-6cd6-498e-a0f8-3b9513f9eaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.423834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0f977-6cd6-498e-a0f8-3b9513f9eaa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.424209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0f977-6cd6-498e-a0f8-3b9513f9eaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.424582 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0f977-6cd6-498e-a0f8-3b9513f9eaa8 admin admin] PATH_INFO: `/v3/projects/4c5e4de3248842728573a525b8e4bb8b/users/e7e9f06e9cac4471bdde18333271972e/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.443295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0f977-6cd6-498e-a0f8-3b9513f9eaa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5e4de3248842728573a525b8e4bb8b, user_id=e7e9f06e9cac4471bdde18333271972e, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.446018 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0f977-6cd6-498e-a0f8-3b9513f9eaa8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.463184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1166/2344] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:24 2022] PUT /identity/v3/projects/4c5e4de3248842728573a525b8e4bb8b/users/e7e9f06e9cac4471bdde18333271972e/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:24.476080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963f89e-90fe-4609-ac39-d5241f798871 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.476668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963f89e-90fe-4609-ac39-d5241f798871 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.477042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963f89e-90fe-4609-ac39-d5241f798871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.509820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3963f89e-90fe-4609-ac39-d5241f798871 None None] MFA Rules not processed for user `e7e9f06e9cac4471bdde18333271972e`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:24.511103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3963f89e-90fe-4609-ac39-d5241f798871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:24.577190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:24.596886 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1179/2345] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:24.598098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0f977-6cd6-498e-a0f8-3b9513f9eaa8 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.611586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:24.680928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0f977-6cd6-498e-a0f8-3b9513f9eaa8 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.685809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc0b9c-97f0-44cf-b512-6a479ccf217e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.686482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc0b9c-97f0-44cf-b512-6a479ccf217e admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.686774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc0b9c-97f0-44cf-b512-6a479ccf217e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.687053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc0b9c-97f0-44cf-b512-6a479ccf217e admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.687797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc0b9c-97f0-44cf-b512-6a479ccf217e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.689747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dc0b9c-97f0-44cf-b512-6a479ccf217e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.729388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1167/2346] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:24 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:24.730615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963f89e-90fe-4609-ac39-d5241f798871 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.745588 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963f89e-90fe-4609-ac39-d5241f798871 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.820409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe56fa4-bc14-490b-b29b-c6a677d7d2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.821614 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe56fa4-bc14-490b-b29b-c6a677d7d2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.822104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe56fa4-bc14-490b-b29b-c6a677d7d2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.822565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe56fa4-bc14-490b-b29b-c6a677d7d2a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.823554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe56fa4-bc14-490b-b29b-c6a677d7d2a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.825695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe56fa4-bc14-490b-b29b-c6a677d7d2a8 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.850423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1180/2347] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:24 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:24.854329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc0b9c-97f0-44cf-b512-6a479ccf217e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.865378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc0b9c-97f0-44cf-b512-6a479ccf217e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.870025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c00d3-caed-4f77-90c6-b445750b1efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.870702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c00d3-caed-4f77-90c6-b445750b1efd admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.870983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c00d3-caed-4f77-90c6-b445750b1efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.871280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975c00d3-caed-4f77-90c6-b445750b1efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.872167 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c00d3-caed-4f77-90c6-b445750b1efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.874385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975c00d3-caed-4f77-90c6-b445750b1efd admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.885872 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1168/2348] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:24.890497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe56fa4-bc14-490b-b29b-c6a677d7d2a8 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.901990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe56fa4-bc14-490b-b29b-c6a677d7d2a8 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:24.906919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa3f23-8a08-49d5-8118-7e0c10160994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:24.907921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa3f23-8a08-49d5-8118-7e0c10160994 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:24.908289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa3f23-8a08-49d5-8118-7e0c10160994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:24.908623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa3f23-8a08-49d5-8118-7e0c10160994 admin admin] PATH_INFO: `/v3/projects/a6e515470697467098cdc314a3b79638/users/bb7bb61bfff24009b962f5023eb5ca99/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:24.934389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa3f23-8a08-49d5-8118-7e0c10160994 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e515470697467098cdc314a3b79638, user_id=bb7bb61bfff24009b962f5023eb5ca99, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:24.937605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa3f23-8a08-49d5-8118-7e0c10160994 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:24.954161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1181/2349] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:24 2022] PUT /identity/v3/projects/a6e515470697467098cdc314a3b79638/users/bb7bb61bfff24009b962f5023eb5ca99/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:19:24.958307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c00d3-caed-4f77-90c6-b445750b1efd admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:24.975460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975c00d3-caed-4f77-90c6-b445750b1efd admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:25.051114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74b420-89f4-47d1-a5da-821d92dae224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:25.051983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b420-89f4-47d1-a5da-821d92dae224 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:25.052273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b420-89f4-47d1-a5da-821d92dae224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:25.052551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74b420-89f4-47d1-a5da-821d92dae224 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:25.053447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74b420-89f4-47d1-a5da-821d92dae224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:25.055533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74b420-89f4-47d1-a5da-821d92dae224 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:25.070711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1169/2350] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:25.074862 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa3f23-8a08-49d5-8118-7e0c10160994 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:25.085812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa3f23-8a08-49d5-8118-7e0c10160994 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:25.090636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:25.091580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:25.091864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:25.092313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] PATH_INFO: `/v3/projects/a6e515470697467098cdc314a3b79638/users/bb7bb61bfff24009b962f5023eb5ca99/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:25.094344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6e515470697467098cdc314a3b79638, user_id=bb7bb61bfff24009b962f5023eb5ca99, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:25.097087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:25.108600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1182/2351] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:25 2022] PUT /identity/v3/projects/a6e515470697467098cdc314a3b79638/users/bb7bb61bfff24009b962f5023eb5ca99/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:19:25.118583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1472a18-d1f3-4de1-99b2-b4ace11fdc82 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:25.118879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1472a18-d1f3-4de1-99b2-b4ace11fdc82 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:25.119161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1472a18-d1f3-4de1-99b2-b4ace11fdc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:25.154253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e1472a18-d1f3-4de1-99b2-b4ace11fdc82 None None] MFA Rules not processed for user `bb7bb61bfff24009b962f5023eb5ca99`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:25.155533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e1472a18-d1f3-4de1-99b2-b4ace11fdc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:25.238489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1170/2352] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:25.251153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:25.253083 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:25.344596 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fcf129-71cb-4afe-9031-05a4bf1eca54 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:25.349746 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fef6a-72fd-4d8e-8593-c99193a5a55d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:25.350765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fef6a-72fd-4d8e-8593-c99193a5a55d service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:25.351210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fef6a-72fd-4d8e-8593-c99193a5a55d service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:25.352033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38fef6a-72fd-4d8e-8593-c99193a5a55d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:25.363677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fef6a-72fd-4d8e-8593-c99193a5a55d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:25.365352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38fef6a-72fd-4d8e-8593-c99193a5a55d service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:25.378659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1183/2353] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:25 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:19:25.498262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1472a18-d1f3-4de1-99b2-b4ace11fdc82 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:25.505227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:25.509791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1472a18-d1f3-4de1-99b2-b4ace11fdc82 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:25.515877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652740cf-26b4-4f17-946e-d1e7b290ab73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:25.517190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652740cf-26b4-4f17-946e-d1e7b290ab73 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:25.518063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652740cf-26b4-4f17-946e-d1e7b290ab73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:25.518405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652740cf-26b4-4f17-946e-d1e7b290ab73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:25.592684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652740cf-26b4-4f17-946e-d1e7b290ab73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:25.594288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652740cf-26b4-4f17-946e-d1e7b290ab73 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:25.615708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1171/2354] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:28.289800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fef6a-72fd-4d8e-8593-c99193a5a55d service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:28.344499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:28.356326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38fef6a-72fd-4d8e-8593-c99193a5a55d service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:28.361967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:28.362867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:28.363144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:28.363445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] PATH_INFO: `/v3/users/e7e9f06e9cac4471bdde18333271972e` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:28.364668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e9f06e9cac4471bdde18333271972e)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:28.366584 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:28.415555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e9f06e9cac4471bdde18333271972e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:28.422480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e9f06e9cac4471bdde18333271972e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:28.423156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e9f06e9cac4471bdde18333271972e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:28.431870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e9f06e9cac4471bdde18333271972e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:28.432534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e9f06e9cac4471bdde18333271972e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:28.452176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1184/2355] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:28 2022] DELETE /identity/v3/users/e7e9f06e9cac4471bdde18333271972e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:28.456788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6defe-2fd3-4573-9a44-3b370de07478 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:28.457324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6defe-2fd3-4573-9a44-3b370de07478 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:28.457499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6defe-2fd3-4573-9a44-3b370de07478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:28.476524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-d2a6defe-2fd3-4573-9a44-3b370de07478 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:28.477749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d2a6defe-2fd3-4573-9a44-3b370de07478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:28.538346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1172/2356] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:28.548506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:28.550060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:28.608890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865950c0-5229-4764-8a70-efadc710a48c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:28.613303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921752cf-276f-4a61-b1eb-4bad94801bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:28.614063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921752cf-276f-4a61-b1eb-4bad94801bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:28.614345 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921752cf-276f-4a61-b1eb-4bad94801bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:28.614609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921752cf-276f-4a61-b1eb-4bad94801bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:28.624177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921752cf-276f-4a61-b1eb-4bad94801bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:28.625715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921752cf-276f-4a61-b1eb-4bad94801bb2 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:28.635742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1185/2357] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:28.907075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6defe-2fd3-4573-9a44-3b370de07478 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:28.908593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d2a6defe-2fd3-4573-9a44-3b370de07478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:28.923562 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6defe-2fd3-4573-9a44-3b370de07478 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:28.928539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:28.929417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:28.929733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:28.930106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] PATH_INFO: `/v3/projects/4c5e4de3248842728573a525b8e4bb8b` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:28.931408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5e4de3248842728573a525b8e4bb8b)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:28.933343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:28.988399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5e4de3248842728573a525b8e4bb8b', 'request_id': 'req-af2834ec-fa48-4ed6-8f30-746e420437de'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:28.997696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5e4de3248842728573a525b8e4bb8b', 'request_id': 'req-af2834ec-fa48-4ed6-8f30-746e420437de'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.003499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5e4de3248842728573a525b8e4bb8b', 'request_id': 'req-af2834ec-fa48-4ed6-8f30-746e420437de'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.014815 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] The token cache is being invalidate because project 4c5e4de3248842728573a525b8e4bb8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:29.015101 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.015864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.017844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1173/2358] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:28 2022] DELETE /identity/v3/projects/4c5e4de3248842728573a525b8e4bb8b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:29.021988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921752cf-276f-4a61-b1eb-4bad94801bb2 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:29.031883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921752cf-276f-4a61-b1eb-4bad94801bb2 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:29.089895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:29.090791 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:29.091091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:29.091395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] PATH_INFO: `/v3/users/bb7bb61bfff24009b962f5023eb5ca99` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:29.092617 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb7bb61bfff24009b962f5023eb5ca99)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:29.094790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:29.108827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:29.146361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb7bb61bfff24009b962f5023eb5ca99'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.153301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb7bb61bfff24009b962f5023eb5ca99'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.153972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb7bb61bfff24009b962f5023eb5ca99'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.164226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb7bb61bfff24009b962f5023eb5ca99'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.165095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-653fa8d5-50bf-4155-9c24-966b3682fc70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb7bb61bfff24009b962f5023eb5ca99'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.183428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1186/2359] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:29 2022] DELETE /identity/v3/users/bb7bb61bfff24009b962f5023eb5ca99 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:29.454349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:29.455928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:29.471860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2834ec-fa48-4ed6-8f30-746e420437de admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:29.497825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:29.531450 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:29.532288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:29.532591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:29.532861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] PATH_INFO: `/v3/projects/a6e515470697467098cdc314a3b79638` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:29.534089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6e515470697467098cdc314a3b79638)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:29.536030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:29.585647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6e515470697467098cdc314a3b79638', 'request_id': 'req-0a558c54-4e94-4360-9b67-59df778d763d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.598246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6e515470697467098cdc314a3b79638', 'request_id': 'req-0a558c54-4e94-4360-9b67-59df778d763d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.605432 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6e515470697467098cdc314a3b79638', 'request_id': 'req-0a558c54-4e94-4360-9b67-59df778d763d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.616831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] The token cache is being invalidate because project a6e515470697467098cdc314a3b79638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:29.617119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.617911 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0a558c54-4e94-4360-9b67-59df778d763d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:29.619728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1174/2360] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:29 2022] DELETE /identity/v3/projects/a6e515470697467098cdc314a3b79638 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:29.625320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562035-08f2-4e2a-b5db-e104e6cdae15 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:29.625840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562035-08f2-4e2a-b5db-e104e6cdae15 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:29.626019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be562035-08f2-4e2a-b5db-e104e6cdae15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:29.646955 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-be562035-08f2-4e2a-b5db-e104e6cdae15 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:29.648221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-be562035-08f2-4e2a-b5db-e104e6cdae15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:29.710086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1187/2361] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:29.723531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3927a2-dfeb-4b00-95b1-cc0a1562ef08 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:29.723531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3927a2-dfeb-4b00-95b1-cc0a1562ef08 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:29.723754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3927a2-dfeb-4b00-95b1-cc0a1562ef08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:29.745241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ea3927a2-dfeb-4b00-95b1-cc0a1562ef08 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:29.746438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ea3927a2-dfeb-4b00-95b1-cc0a1562ef08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:29.754435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1175/2362] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:29.758840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be562035-08f2-4e2a-b5db-e104e6cdae15 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:29.769497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be562035-08f2-4e2a-b5db-e104e6cdae15 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:29.774798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:29.775489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:29.775775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:29.776047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:29.776782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:19:29.777085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:29.778717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:29.790710 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1188/2363] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:19:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:19:29.795840 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3927a2-dfeb-4b00-95b1-cc0a1562ef08 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:29.811141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3927a2-dfeb-4b00-95b1-cc0a1562ef08 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:29.820058 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741d9a1-e8e6-429b-9078-4dab4eea914c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:29.821001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741d9a1-e8e6-429b-9078-4dab4eea914c admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:29.821381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741d9a1-e8e6-429b-9078-4dab4eea914c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:29.821748 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b741d9a1-e8e6-429b-9078-4dab4eea914c admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:29.822741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741d9a1-e8e6-429b-9078-4dab4eea914c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:29.825816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b741d9a1-e8e6-429b-9078-4dab4eea914c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:29.854721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1176/2364] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:29 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:29.859480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:29.870108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5bdceb-c6d9-4bc1-a3f0-24f6ca0a6f31 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:29.926518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40cd0c-e62a-4309-8cac-a4dcb70727d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:29.927215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40cd0c-e62a-4309-8cac-a4dcb70727d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:29.927532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40cd0c-e62a-4309-8cac-a4dcb70727d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:29.927807 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab40cd0c-e62a-4309-8cac-a4dcb70727d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:29.928498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40cd0c-e62a-4309-8cac-a4dcb70727d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:29.930418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab40cd0c-e62a-4309-8cac-a4dcb70727d5 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:29.952981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1189/2365] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:29 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:29.957268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741d9a1-e8e6-429b-9078-4dab4eea914c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:29.968134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b741d9a1-e8e6-429b-9078-4dab4eea914c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:29.973201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3253044-010d-47a3-bd28-7c8718d2a3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:29.973890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253044-010d-47a3-bd28-7c8718d2a3cd admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:29.974177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253044-010d-47a3-bd28-7c8718d2a3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:29.974442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3253044-010d-47a3-bd28-7c8718d2a3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:29.975216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3253044-010d-47a3-bd28-7c8718d2a3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:29.976875 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3253044-010d-47a3-bd28-7c8718d2a3cd admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:29.987675 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1177/2366] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:29.993266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40cd0c-e62a-4309-8cac-a4dcb70727d5 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.004107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab40cd0c-e62a-4309-8cac-a4dcb70727d5 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.009172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f99dc9-ce91-4927-a11a-f421b99b3ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.009835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f99dc9-ce91-4927-a11a-f421b99b3ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.010122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f99dc9-ce91-4927-a11a-f421b99b3ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.010403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f99dc9-ce91-4927-a11a-f421b99b3ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.011156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f99dc9-ce91-4927-a11a-f421b99b3ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.012852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f99dc9-ce91-4927-a11a-f421b99b3ab2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.015144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:30.026214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1190/2367] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:30.030501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3253044-010d-47a3-bd28-7c8718d2a3cd admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.040566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3253044-010d-47a3-bd28-7c8718d2a3cd admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.045751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2e132-c5d3-458e-95fd-7b5adf0de042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.046768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2e132-c5d3-458e-95fd-7b5adf0de042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.047062 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2e132-c5d3-458e-95fd-7b5adf0de042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.047388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b2e132-c5d3-458e-95fd-7b5adf0de042 admin admin] PATH_INFO: `/v3/projects/3a6f9eff12984511b51ba1902a9df0c6/users/35efc3b9b3704f5eba972997e7079104/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.061913 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2e132-c5d3-458e-95fd-7b5adf0de042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a6f9eff12984511b51ba1902a9df0c6, user_id=35efc3b9b3704f5eba972997e7079104, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.064753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b2e132-c5d3-458e-95fd-7b5adf0de042 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.080967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1178/2368] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:30 2022] PUT /identity/v3/projects/3a6f9eff12984511b51ba1902a9df0c6/users/35efc3b9b3704f5eba972997e7079104/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:30.092167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a3bc0-b5c2-4f6b-b017-5c287677291d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.092457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a3bc0-b5c2-4f6b-b017-5c287677291d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.092777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21a3bc0-b5c2-4f6b-b017-5c287677291d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.120165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a21a3bc0-b5c2-4f6b-b017-5c287677291d None None] MFA Rules not processed for user `35efc3b9b3704f5eba972997e7079104`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:30.121319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a21a3bc0-b5c2-4f6b-b017-5c287677291d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:30.188023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1191/2369] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:30.193125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2e132-c5d3-458e-95fd-7b5adf0de042 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.250892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:30.264596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b2e132-c5d3-458e-95fd-7b5adf0de042 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.270448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c6e54-901e-4dd7-b3d7-faa152b4c069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.271115 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c6e54-901e-4dd7-b3d7-faa152b4c069 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.271431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c6e54-901e-4dd7-b3d7-faa152b4c069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.271710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c6e54-901e-4dd7-b3d7-faa152b4c069 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.272409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c6e54-901e-4dd7-b3d7-faa152b4c069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.274327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271c6e54-901e-4dd7-b3d7-faa152b4c069 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.307456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1179/2370] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:30 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:30.312386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a3bc0-b5c2-4f6b-b017-5c287677291d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.327662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21a3bc0-b5c2-4f6b-b017-5c287677291d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.386024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca670f-ee4a-4d5a-a75a-91530a3756dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.386698 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca670f-ee4a-4d5a-a75a-91530a3756dd admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.387003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca670f-ee4a-4d5a-a75a-91530a3756dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.387283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecca670f-ee4a-4d5a-a75a-91530a3756dd admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.387996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca670f-ee4a-4d5a-a75a-91530a3756dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.389967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecca670f-ee4a-4d5a-a75a-91530a3756dd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.413586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1192/2371] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:30 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:19:30.417587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c6e54-901e-4dd7-b3d7-faa152b4c069 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.428717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c6e54-901e-4dd7-b3d7-faa152b4c069 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.433939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059dfd9-46f0-41bd-9123-7755ead9bbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.434610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059dfd9-46f0-41bd-9123-7755ead9bbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.434890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059dfd9-46f0-41bd-9123-7755ead9bbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.435194 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059dfd9-46f0-41bd-9123-7755ead9bbb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.436159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059dfd9-46f0-41bd-9123-7755ead9bbb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.437783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059dfd9-46f0-41bd-9123-7755ead9bbb4 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.448707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1180/2372] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:30.453164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca670f-ee4a-4d5a-a75a-91530a3756dd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.463816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecca670f-ee4a-4d5a-a75a-91530a3756dd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.468477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd58d3-5d50-4dd3-8d47-a1ff098e6c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.469390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd58d3-5d50-4dd3-8d47-a1ff098e6c4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.469673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd58d3-5d50-4dd3-8d47-a1ff098e6c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.469948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbd58d3-5d50-4dd3-8d47-a1ff098e6c4c admin admin] PATH_INFO: `/v3/projects/01787abb2bb946118086efe6324bd016/users/e685a10cd198484ea36b4dc7e33aefc9/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.485416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd58d3-5d50-4dd3-8d47-a1ff098e6c4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01787abb2bb946118086efe6324bd016, user_id=e685a10cd198484ea36b4dc7e33aefc9, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.488323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbd58d3-5d50-4dd3-8d47-a1ff098e6c4c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.504571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1193/2373] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:30 2022] PUT /identity/v3/projects/01787abb2bb946118086efe6324bd016/users/e685a10cd198484ea36b4dc7e33aefc9/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:30.509269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059dfd9-46f0-41bd-9123-7755ead9bbb4 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.521364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059dfd9-46f0-41bd-9123-7755ead9bbb4 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.596416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b513a2-d6aa-4c99-8e70-403fa1f2c2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.597097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b513a2-d6aa-4c99-8e70-403fa1f2c2df admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.597378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b513a2-d6aa-4c99-8e70-403fa1f2c2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.597686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b513a2-d6aa-4c99-8e70-403fa1f2c2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.598517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b513a2-d6aa-4c99-8e70-403fa1f2c2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.600311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b513a2-d6aa-4c99-8e70-403fa1f2c2df admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.611181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1181/2374] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:30 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:30.615449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd58d3-5d50-4dd3-8d47-a1ff098e6c4c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.626340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbd58d3-5d50-4dd3-8d47-a1ff098e6c4c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.631690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.632650 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.633071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.633303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] PATH_INFO: `/v3/projects/01787abb2bb946118086efe6324bd016/users/e685a10cd198484ea36b4dc7e33aefc9/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.635325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01787abb2bb946118086efe6324bd016, user_id=e685a10cd198484ea36b4dc7e33aefc9, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.638126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.649369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1194/2375] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:30 2022] PUT /identity/v3/projects/01787abb2bb946118086efe6324bd016/users/e685a10cd198484ea36b4dc7e33aefc9/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:30.659716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6966-c049-4896-9bab-bfac1e2ba536 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.660008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6966-c049-4896-9bab-bfac1e2ba536 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.660313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a6966-c049-4896-9bab-bfac1e2ba536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.686670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-1d2a6966-c049-4896-9bab-bfac1e2ba536 None None] MFA Rules not processed for user `e685a10cd198484ea36b4dc7e33aefc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:30.687909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1d2a6966-c049-4896-9bab-bfac1e2ba536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:30.759199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1182/2376] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:30 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:30.779169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:30.781025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:30.831990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:30.845628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4dce72-ad74-4f7b-8444-c6dac7949701 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:30.850003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:30.850654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:30.850931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:30.851410 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:30.912914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:30.914398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:30.925049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1195/2377] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:33.020105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475702b-b8c5-42b1-81b2-5f24a3d6ee1b None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:33.020602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475702b-b8c5-42b1-81b2-5f24a3d6ee1b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:33.020707 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475702b-b8c5-42b1-81b2-5f24a3d6ee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:33.043280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-1475702b-b8c5-42b1-81b2-5f24a3d6ee1b None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:33.044427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1475702b-b8c5-42b1-81b2-5f24a3d6ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:33.092699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:33.105932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1183/2378] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:33.118324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:33.119828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:33.191546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a4c93-520f-4ce6-9251-15f01de000d0 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:33.197176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef079b-de33-4d50-9739-afa0909f1875 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:33.197828 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef079b-de33-4d50-9739-afa0909f1875 service swift] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:33.198115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef079b-de33-4d50-9739-afa0909f1875 service swift] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:33.198392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef079b-de33-4d50-9739-afa0909f1875 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:33.207239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef079b-de33-4d50-9739-afa0909f1875 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:33.208903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef079b-de33-4d50-9739-afa0909f1875 service swift] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:33.217598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1196/2379] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:19:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:19:33.485535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475702b-b8c5-42b1-81b2-5f24a3d6ee1b None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:33.566262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475702b-b8c5-42b1-81b2-5f24a3d6ee1b None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:33.570977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:33.571869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:33.572327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:33.572607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] PATH_INFO: `/v3/users/35efc3b9b3704f5eba972997e7079104` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:33.573847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35efc3b9b3704f5eba972997e7079104)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:33.576006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:33.641857 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35efc3b9b3704f5eba972997e7079104'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:33.642887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35efc3b9b3704f5eba972997e7079104'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:33.643543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35efc3b9b3704f5eba972997e7079104'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:33.656983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35efc3b9b3704f5eba972997e7079104'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:33.674783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35efc3b9b3704f5eba972997e7079104'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:33.707292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1184/2380] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:33 2022] DELETE /identity/v3/users/35efc3b9b3704f5eba972997e7079104 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:33.711825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7ad3e-72e2-4feb-910a-6d6e9a5ed3c5 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:33.712404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7ad3e-72e2-4feb-910a-6d6e9a5ed3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:33.712879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a7ad3e-72e2-4feb-910a-6d6e9a5ed3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:33.736372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b0a7ad3e-72e2-4feb-910a-6d6e9a5ed3c5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:33.737792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b0a7ad3e-72e2-4feb-910a-6d6e9a5ed3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:33.831878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1197/2381] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:19:33.842887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:33.844501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:33.910373 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c84ec2c-592b-4496-a0d4-79a11425f955 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:33.914527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a45cdf-5427-44ef-b2c0-90ccc6ac3fcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:33.915383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a45cdf-5427-44ef-b2c0-90ccc6ac3fcd service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:33.915710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a45cdf-5427-44ef-b2c0-90ccc6ac3fcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:33.915990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a45cdf-5427-44ef-b2c0-90ccc6ac3fcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:33.925996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a45cdf-5427-44ef-b2c0-90ccc6ac3fcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:33.927497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a45cdf-5427-44ef-b2c0-90ccc6ac3fcd service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:33.938365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1185/2382] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:34.229694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7ad3e-72e2-4feb-910a-6d6e9a5ed3c5 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:34.232088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b0a7ad3e-72e2-4feb-910a-6d6e9a5ed3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:34.244819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a7ad3e-72e2-4feb-910a-6d6e9a5ed3c5 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:34.249689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:34.250527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:34.250830 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:34.251129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] PATH_INFO: `/v3/projects/3a6f9eff12984511b51ba1902a9df0c6` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:34.252320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a6f9eff12984511b51ba1902a9df0c6)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:34.254221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:34.280643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:34.319507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a6f9eff12984511b51ba1902a9df0c6', 'request_id': 'req-9528ad2f-d530-46aa-95d7-708091b6635f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.328689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a6f9eff12984511b51ba1902a9df0c6', 'request_id': 'req-9528ad2f-d530-46aa-95d7-708091b6635f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.335962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a6f9eff12984511b51ba1902a9df0c6', 'request_id': 'req-9528ad2f-d530-46aa-95d7-708091b6635f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.352781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] The token cache is being invalidate because project 3a6f9eff12984511b51ba1902a9df0c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:34.353138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.353756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.355639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1198/2383] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:34 2022] DELETE /identity/v3/projects/3a6f9eff12984511b51ba1902a9df0c6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:34.359643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a45cdf-5427-44ef-b2c0-90ccc6ac3fcd service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:34.370035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a45cdf-5427-44ef-b2c0-90ccc6ac3fcd service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:34.382637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:34.431278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:34.432189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:34.432470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:34.432743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] PATH_INFO: `/v3/users/e685a10cd198484ea36b4dc7e33aefc9` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:34.434034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e685a10cd198484ea36b4dc7e33aefc9)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:34.436563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:34.488285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e685a10cd198484ea36b4dc7e33aefc9'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.489059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e685a10cd198484ea36b4dc7e33aefc9'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.489757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e685a10cd198484ea36b4dc7e33aefc9'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.503556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e685a10cd198484ea36b4dc7e33aefc9'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.511768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d48ce5b-22f8-4564-a805-8d7c64e9690f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e685a10cd198484ea36b4dc7e33aefc9'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:34.534370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1186/2384] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:34 2022] DELETE /identity/v3/users/e685a10cd198484ea36b4dc7e33aefc9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:34.825903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:34.826615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:34.847360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528ad2f-d530-46aa-95d7-708091b6635f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:34.939130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:34.939958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:34.940239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:34.940514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] PATH_INFO: `/v3/projects/01787abb2bb946118086efe6324bd016` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:34.941722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01787abb2bb946118086efe6324bd016)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:34.943915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:34.993222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01787abb2bb946118086efe6324bd016', 'request_id': 'req-3464f443-bfac-43ab-ae18-37f782958bf6'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:35.000434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01787abb2bb946118086efe6324bd016', 'request_id': 'req-3464f443-bfac-43ab-ae18-37f782958bf6'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:35.007214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01787abb2bb946118086efe6324bd016', 'request_id': 'req-3464f443-bfac-43ab-ae18-37f782958bf6'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:35.032108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] The token cache is being invalidate because project 01787abb2bb946118086efe6324bd016 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:35.032429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:35.033100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3464f443-bfac-43ab-ae18-37f782958bf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:35.034714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1199/2385] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:34 2022] DELETE /identity/v3/projects/01787abb2bb946118086efe6324bd016 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:35.039667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c5d4c-1003-4b80-862b-096d04fcd5db None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.039976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c5d4c-1003-4b80-862b-096d04fcd5db None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.040251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91c5d4c-1003-4b80-862b-096d04fcd5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.060896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b91c5d4c-1003-4b80-862b-096d04fcd5db None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:35.062449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b91c5d4c-1003-4b80-862b-096d04fcd5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:35.124257 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1187/2386] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:35.133176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4fe0a-f945-49a3-8a7e-22b0a74303dd None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.133467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4fe0a-f945-49a3-8a7e-22b0a74303dd None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.133750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4fe0a-f945-49a3-8a7e-22b0a74303dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.153580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-0bf4fe0a-f945-49a3-8a7e-22b0a74303dd None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:35.155034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0bf4fe0a-f945-49a3-8a7e-22b0a74303dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:35.163123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1200/2387] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:35.167637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c5d4c-1003-4b80-862b-096d04fcd5db None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:35.179036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91c5d4c-1003-4b80-862b-096d04fcd5db None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:35.185834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:35.186569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.186849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.187124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.187880 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:19:35.188196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:35.189863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:35.202008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1188/2388] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:19:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:19:35.206395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4fe0a-f945-49a3-8a7e-22b0a74303dd None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:35.216510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4fe0a-f945-49a3-8a7e-22b0a74303dd None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:35.221238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460c581-0cf4-459e-b79b-1e10b48b1916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:35.221949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c581-0cf4-459e-b79b-1e10b48b1916 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.222223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c581-0cf4-459e-b79b-1e10b48b1916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.222528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4460c581-0cf4-459e-b79b-1e10b48b1916 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.223228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460c581-0cf4-459e-b79b-1e10b48b1916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:35.225177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4460c581-0cf4-459e-b79b-1e10b48b1916 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:35.260736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:35.278882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1201/2389] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:35 2022] POST /identity/v3/projects => generated 375 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:35.281378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:35.292503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee750c1f-bfc5-4a67-ba50-45cd333af406 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:35.313841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:35.355781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1784-6b4d-46e9-a535-57c595a8144a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:35.356472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1784-6b4d-46e9-a535-57c595a8144a admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.356760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1784-6b4d-46e9-a535-57c595a8144a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.357036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa1784-6b4d-46e9-a535-57c595a8144a admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.357754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa1784-6b4d-46e9-a535-57c595a8144a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:35.360092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa1784-6b4d-46e9-a535-57c595a8144a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:35.386225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1189/2390] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:35 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:19:35.387976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460c581-0cf4-459e-b79b-1e10b48b1916 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:35.404806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4460c581-0cf4-459e-b79b-1e10b48b1916 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:35.411096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea190d7e-bc9d-4387-b1b1-310f731af6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:35.411915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea190d7e-bc9d-4387-b1b1-310f731af6e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.412197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea190d7e-bc9d-4387-b1b1-310f731af6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.412472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea190d7e-bc9d-4387-b1b1-310f731af6e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.413218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea190d7e-bc9d-4387-b1b1-310f731af6e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:35.414900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea190d7e-bc9d-4387-b1b1-310f731af6e2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:35.425529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1202/2391] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:35.429797 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1784-6b4d-46e9-a535-57c595a8144a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:35.439836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa1784-6b4d-46e9-a535-57c595a8144a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:35.444603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c58de5-cf60-4d50-a892-e94ef564028a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:35.445198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c58de5-cf60-4d50-a892-e94ef564028a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.445477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c58de5-cf60-4d50-a892-e94ef564028a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.445749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c58de5-cf60-4d50-a892-e94ef564028a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.446483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c58de5-cf60-4d50-a892-e94ef564028a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:35.448256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c58de5-cf60-4d50-a892-e94ef564028a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:35.458943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1190/2392] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:35.464360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea190d7e-bc9d-4387-b1b1-310f731af6e2 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:35.475200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea190d7e-bc9d-4387-b1b1-310f731af6e2 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:35.480098 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:35.481013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.481296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.481572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] PATH_INFO: `/v3/projects/62349497689048dab7f1fcdfbbe08d37/users/074c96e6d2214db8b68b85cc0bc00326/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.498000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=62349497689048dab7f1fcdfbbe08d37, user_id=074c96e6d2214db8b68b85cc0bc00326, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:35.500800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:35.515605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1203/2393] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:35 2022] PUT /identity/v3/projects/62349497689048dab7f1fcdfbbe08d37/users/074c96e6d2214db8b68b85cc0bc00326/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:35.526469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a541-b1f0-4175-8268-61763ddff986 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.526763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a541-b1f0-4175-8268-61763ddff986 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.527045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a5a541-b1f0-4175-8268-61763ddff986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.559490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e4a5a541-b1f0-4175-8268-61763ddff986 None None] MFA Rules not processed for user `074c96e6d2214db8b68b85cc0bc00326`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:35.563072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e4a5a541-b1f0-4175-8268-61763ddff986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:35.632954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1191/2394] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:35.646250 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:35.648151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:35.715788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98374933-88ea-4dc6-a3bf-cc791526df2d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:35.719529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca58dd6-65c1-4a82-aa7a-e0281bdfbd75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:35.720265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca58dd6-65c1-4a82-aa7a-e0281bdfbd75 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:35.720638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca58dd6-65c1-4a82-aa7a-e0281bdfbd75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:35.721110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca58dd6-65c1-4a82-aa7a-e0281bdfbd75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:35.730848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca58dd6-65c1-4a82-aa7a-e0281bdfbd75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:35.732387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca58dd6-65c1-4a82-aa7a-e0281bdfbd75 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:35.741536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1204/2395] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:38.932577 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5a541-b1f0-4175-8268-61763ddff986 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:38.996838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a5a541-b1f0-4175-8268-61763ddff986 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:39.001853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:39.002743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:39.003159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:39.003497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] PATH_INFO: `/v3/users/074c96e6d2214db8b68b85cc0bc00326` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:39.004730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=074c96e6d2214db8b68b85cc0bc00326)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:39.006719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:39.052917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:39.073672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '074c96e6d2214db8b68b85cc0bc00326'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.074540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '074c96e6d2214db8b68b85cc0bc00326'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.075347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '074c96e6d2214db8b68b85cc0bc00326'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.088268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '074c96e6d2214db8b68b85cc0bc00326'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.100030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '074c96e6d2214db8b68b85cc0bc00326'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.120956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1192/2396] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:19:38 2022] DELETE /identity/v3/users/074c96e6d2214db8b68b85cc0bc00326 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:39.125755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6197ab5-467b-46f6-a785-43ce6f5450e8 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:39.126355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6197ab5-467b-46f6-a785-43ce6f5450e8 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:39.126355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6197ab5-467b-46f6-a785-43ce6f5450e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:39.157336 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e6197ab5-467b-46f6-a785-43ce6f5450e8 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:39.158573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e6197ab5-467b-46f6-a785-43ce6f5450e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:39.244685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1205/2397] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:39.255130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:39.256702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:39.347904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb25b8-7122-4001-9881-30ceb8263a26 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:39.351886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa959c62-ec46-4211-92d6-d8964e44c3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:39.352586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959c62-ec46-4211-92d6-d8964e44c3dd service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:39.352871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959c62-ec46-4211-92d6-d8964e44c3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:39.353168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa959c62-ec46-4211-92d6-d8964e44c3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:39.365712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959c62-ec46-4211-92d6-d8964e44c3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:39.367334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa959c62-ec46-4211-92d6-d8964e44c3dd service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:39.378465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1193/2398] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:39.754514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6197ab5-467b-46f6-a785-43ce6f5450e8 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:39.756015 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e6197ab5-467b-46f6-a785-43ce6f5450e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:39.783062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6197ab5-467b-46f6-a785-43ce6f5450e8 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:39.788537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:39.789484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:39.789826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:39.790547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] PATH_INFO: `/v3/projects/62349497689048dab7f1fcdfbbe08d37` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:39.791881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=62349497689048dab7f1fcdfbbe08d37)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:39.793870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:39.876295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62349497689048dab7f1fcdfbbe08d37', 'request_id': 'req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.883817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62349497689048dab7f1fcdfbbe08d37', 'request_id': 'req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.891089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62349497689048dab7f1fcdfbbe08d37', 'request_id': 'req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.893643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:39.916548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] The token cache is being invalidate because project 62349497689048dab7f1fcdfbbe08d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:19:39.916917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.917556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-1b52cb26-bbf6-4563-915d-0ed6a1b762bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:19:39.923751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1206/2399] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:19:39 2022] DELETE /identity/v3/projects/62349497689048dab7f1fcdfbbe08d37 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:39.925845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71ce4-ae16-4efa-b068-5c5091ec1bb5 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:39.926140 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71ce4-ae16-4efa-b068-5c5091ec1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:39.926421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c71ce4-ae16-4efa-b068-5c5091ec1bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:39.945643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-10c71ce4-ae16-4efa-b068-5c5091ec1bb5 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:39.946869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-10c71ce4-ae16-4efa-b068-5c5091ec1bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:40.016388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1194/2400] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:40.024279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c5d12-db1f-4dd4-a8c1-15e980975890 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.024569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c5d12-db1f-4dd4-a8c1-15e980975890 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.024848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c5d12-db1f-4dd4-a8c1-15e980975890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.051724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-e95c5d12-db1f-4dd4-a8c1-15e980975890 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:40.051724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e95c5d12-db1f-4dd4-a8c1-15e980975890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:40.059304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1207/2401] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:40.061739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71ce4-ae16-4efa-b068-5c5091ec1bb5 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:40.072241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c71ce4-ae16-4efa-b068-5c5091ec1bb5 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:40.078002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:40.078756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.079040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.079407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.080232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:19:40.080614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:40.083172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:40.096046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1195/2402] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:19:40.100645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c5d12-db1f-4dd4-a8c1-15e980975890 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:40.109859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c5d12-db1f-4dd4-a8c1-15e980975890 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:40.114594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9db3c-51c8-4d14-8894-4380165d952f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:40.115282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9db3c-51c8-4d14-8894-4380165d952f admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.115583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9db3c-51c8-4d14-8894-4380165d952f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.115858 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9db3c-51c8-4d14-8894-4380165d952f admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.116584 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9db3c-51c8-4d14-8894-4380165d952f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:40.118501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9db3c-51c8-4d14-8894-4380165d952f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:40.153166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1208/2403] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:19:40 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:40.157443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:40.168273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9cbcac-a93c-453e-b6f2-a42c0705b685 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:40.210945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:40.229190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154551f-3abf-45d7-b5c5-5b9d4b62276c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:40.229885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154551f-3abf-45d7-b5c5-5b9d4b62276c admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.230170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154551f-3abf-45d7-b5c5-5b9d4b62276c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.230448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154551f-3abf-45d7-b5c5-5b9d4b62276c admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.231294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154551f-3abf-45d7-b5c5-5b9d4b62276c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:40.233336 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154551f-3abf-45d7-b5c5-5b9d4b62276c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:40.257159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1196/2404] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:19:40 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:19:40.261287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9db3c-51c8-4d14-8894-4380165d952f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:40.272223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9db3c-51c8-4d14-8894-4380165d952f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:40.276854 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793ac6-b263-4b3f-94d5-665eb7954410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:40.277518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793ac6-b263-4b3f-94d5-665eb7954410 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.277796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793ac6-b263-4b3f-94d5-665eb7954410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.278070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793ac6-b263-4b3f-94d5-665eb7954410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.278801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9793ac6-b263-4b3f-94d5-665eb7954410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:40.280461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9793ac6-b263-4b3f-94d5-665eb7954410 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:40.291212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1209/2405] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:40.295586 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154551f-3abf-45d7-b5c5-5b9d4b62276c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:40.307978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154551f-3abf-45d7-b5c5-5b9d4b62276c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:40.314680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4cc57-4d94-4d86-8b70-872278e06418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:40.315616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4cc57-4d94-4d86-8b70-872278e06418 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.315750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4cc57-4d94-4d86-8b70-872278e06418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.315966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4cc57-4d94-4d86-8b70-872278e06418 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.316822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4cc57-4d94-4d86-8b70-872278e06418 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:40.318462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4cc57-4d94-4d86-8b70-872278e06418 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:40.330807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1197/2406] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:19:40.335082 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793ac6-b263-4b3f-94d5-665eb7954410 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:40.345709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793ac6-b263-4b3f-94d5-665eb7954410 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:40.351219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:40.352367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.352813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.353249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] PATH_INFO: `/v3/projects/e619f7d90e8543ba85f3bbaba9046c72/users/661271a657a548468ff85fe298e0c5f9/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.370907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e619f7d90e8543ba85f3bbaba9046c72, user_id=661271a657a548468ff85fe298e0c5f9, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:40.373645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:40.389475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1210/2407] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:19:40 2022] PUT /identity/v3/projects/e619f7d90e8543ba85f3bbaba9046c72/users/661271a657a548468ff85fe298e0c5f9/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:19:40.395752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9afc7c-3ad2-4b4e-979f-0955c1691be8 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.396133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9afc7c-3ad2-4b4e-979f-0955c1691be8 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.396477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9afc7c-3ad2-4b4e-979f-0955c1691be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.416068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0c9afc7c-3ad2-4b4e-979f-0955c1691be8 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:40.417429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0c9afc7c-3ad2-4b4e-979f-0955c1691be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:40.478311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1198/2408] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:40.488264 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:40.490186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:40.565835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f899b-1e5f-45c8-87c4-d0e2e839edd1 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:40.569847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:40.570509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:40.570793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:40.571067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:40.580421 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:40.581852 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:40.591440 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1211/2409] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:41.510774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3cdc-b63c-480b-8637-5a621c1c3e15 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:41.511470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3cdc-b63c-480b-8637-5a621c1c3e15 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:41.511470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db3cdc-b63c-480b-8637-5a621c1c3e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:41.540425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-a5db3cdc-b63c-480b-8637-5a621c1c3e15 None None] MFA Rules not processed for user `661271a657a548468ff85fe298e0c5f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:41.546221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a5db3cdc-b63c-480b-8637-5a621c1c3e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:41.639300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1199/2410] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:41.651388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:41.652815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:41.722709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35bef19-7758-46b3-908e-374be9e9e8a0 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:41.726225 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e12b6-cba8-4e12-98d3-01c78ff60bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:41.726868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e12b6-cba8-4e12-98d3-01c78ff60bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:41.727147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e12b6-cba8-4e12-98d3-01c78ff60bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:41.727522 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e12b6-cba8-4e12-98d3-01c78ff60bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:41.735775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e12b6-cba8-4e12-98d3-01c78ff60bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:41.737198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e12b6-cba8-4e12-98d3-01c78ff60bb8 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:41.745716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1212/2411] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:19:43.188275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc8a26-f6e3-4929-b3bf-d87faaf85742 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:43.188825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc8a26-f6e3-4929-b3bf-d87faaf85742 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:43.188825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc8a26-f6e3-4929-b3bf-d87faaf85742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:43.207138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9cfc8a26-f6e3-4929-b3bf-d87faaf85742 None None] MFA Rules not processed for user `661271a657a548468ff85fe298e0c5f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:43.208355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9cfc8a26-f6e3-4929-b3bf-d87faaf85742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:43.216223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1200/2412] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:43.446370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e12b6-cba8-4e12-98d3-01c78ff60bb8 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:43.456205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e12b6-cba8-4e12-98d3-01c78ff60bb8 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:43.460162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016510eb-82e7-44e1-aefd-7206ac6ea04c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:43.460792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016510eb-82e7-44e1-aefd-7206ac6ea04c service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:43.461075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016510eb-82e7-44e1-aefd-7206ac6ea04c service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:43.461344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016510eb-82e7-44e1-aefd-7206ac6ea04c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:43.463253 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-016510eb-82e7-44e1-aefd-7206ac6ea04c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:43.473031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016510eb-82e7-44e1-aefd-7206ac6ea04c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:43.474452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016510eb-82e7-44e1-aefd-7206ac6ea04c service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:43.482487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1213/2413] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:46.909315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc8a26-f6e3-4929-b3bf-d87faaf85742 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:46.911185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9cfc8a26-f6e3-4929-b3bf-d87faaf85742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:47.004611 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:47.038331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc8a26-f6e3-4929-b3bf-d87faaf85742 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:47.044629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33200547-74d8-4c81-b673-7e8a68760e7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:47.045448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200547-74d8-4c81-b673-7e8a68760e7d service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:47.045806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200547-74d8-4c81-b673-7e8a68760e7d service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:47.046193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33200547-74d8-4c81-b673-7e8a68760e7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:47.048387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-33200547-74d8-4c81-b673-7e8a68760e7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:47.062298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33200547-74d8-4c81-b673-7e8a68760e7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:47.064005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33200547-74d8-4c81-b673-7e8a68760e7d service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:47.077823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1201/2414] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:19:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:19:47.783722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a5a4f-9022-4a41-874a-5d4c0a283479 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:47.784308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a5a4f-9022-4a41-874a-5d4c0a283479 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:47.784308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3a5a4f-9022-4a41-874a-5d4c0a283479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:47.817328 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-1f3a5a4f-9022-4a41-874a-5d4c0a283479 None None] MFA Rules not processed for user `bc0307909d614a1f97f006dc6d091cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:47.834123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1f3a5a4f-9022-4a41-874a-5d4c0a283479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:47.878353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:19:47.983156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1214/2415] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:19:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:49.945535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7e67-4f5f-4c7e-9ad0-3465f6f5f0a2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:49.946098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7e67-4f5f-4c7e-9ad0-3465f6f5f0a2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:49.946197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6d7e67-4f5f-4c7e-9ad0-3465f6f5f0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:49.979198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-5a6d7e67-4f5f-4c7e-9ad0-3465f6f5f0a2 None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:19:49.980433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5a6d7e67-4f5f-4c7e-9ad0-3465f6f5f0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:50.038941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1202/2416] 10.222.0.69 () {58 vars in 967 bytes} [Fri May 20 09:19:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:19:50.053076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a5a4f-9022-4a41-874a-5d4c0a283479 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:19:50.054520 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1f3a5a4f-9022-4a41-874a-5d4c0a283479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:19:50.127012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3a5a4f-9022-4a41-874a-5d4c0a283479 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:19:50.132923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:19:50.133834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:19:50.134129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:19:50.134409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:19:50.146382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:19:50.148039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:19:50.156232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1215/2417] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:19:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:20:00.031103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=40 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:00.152893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=48 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:07.837518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d7e67-4f5f-4c7e-9ad0-3465f6f5f0a2 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:07.847116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6d7e67-4f5f-4c7e-9ad0-3465f6f5f0a2 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:07.850951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:07.851833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:07.852188 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:07.852579 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:07.854508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:07.867793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:07.869261 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:07.879178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1203/2418] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:11.607033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:11.609129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:11.707600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f830d70a-d1fa-493c-a5de-c1e2ba658ffc service swift] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:11.711501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:11.712749 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:11.713035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:11.713236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:11.713680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:11.714772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:11.715848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:11.729018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:11.730748 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:11.738046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001d747-5ad3-48c9-a445-36e2efeba1c4 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:11.741572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1216/2419] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:11.743189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:11.746410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:11.747032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:11.747567 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] PATH_INFO: `/v3/users/ff944e4cf29b4297acc63b591d90daaf` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:11.749208 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff944e4cf29b4297acc63b591d90daaf)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:11.751609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:11.811400 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff944e4cf29b4297acc63b591d90daaf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:11.812102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff944e4cf29b4297acc63b591d90daaf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:11.812790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff944e4cf29b4297acc63b591d90daaf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:11.821756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff944e4cf29b4297acc63b591d90daaf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:11.829030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7d52c260-85e9-4b13-bdd4-b489c7e62025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff944e4cf29b4297acc63b591d90daaf'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:11.846232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1204/2420] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:11 2022] DELETE /identity/v3/users/ff944e4cf29b4297acc63b591d90daaf => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:12.019488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:12.030615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411b91f9-701a-4e4d-b145-cc40e902bfe6 service placement] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:12.089194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:12.090693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.091174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.091701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] PATH_INFO: `/v3/projects/2ddfd22e3dde4cc89c073bd2d88d648b` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.093504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ddfd22e3dde4cc89c073bd2d88d648b)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:12.095536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:12.145534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ddfd22e3dde4cc89c073bd2d88d648b', 'request_id': 'req-a4571837-b12b-439f-a24d-f98a119d53ca'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:12.150889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ddfd22e3dde4cc89c073bd2d88d648b', 'request_id': 'req-a4571837-b12b-439f-a24d-f98a119d53ca'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:12.157115 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ddfd22e3dde4cc89c073bd2d88d648b', 'request_id': 'req-a4571837-b12b-439f-a24d-f98a119d53ca'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:12.172487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] The token cache is being invalidate because project 2ddfd22e3dde4cc89c073bd2d88d648b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:12.172804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:12.173431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4571837-b12b-439f-a24d-f98a119d53ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:12.175129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1217/2421] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:12 2022] DELETE /identity/v3/projects/2ddfd22e3dde4cc89c073bd2d88d648b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:12.180974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce6472-fa8b-40f8-b199-ed5e921c99de None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.180974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce6472-fa8b-40f8-b199-ed5e921c99de None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.180974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edce6472-fa8b-40f8-b199-ed5e921c99de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.204852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-edce6472-fa8b-40f8-b199-ed5e921c99de None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:12.205864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-edce6472-fa8b-40f8-b199-ed5e921c99de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:12.269662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1205/2422] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:12.278344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a796bb7-0522-4e75-9117-cae0e898270d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.278633 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a796bb7-0522-4e75-9117-cae0e898270d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.278913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a796bb7-0522-4e75-9117-cae0e898270d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.297069 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4a796bb7-0522-4e75-9117-cae0e898270d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:12.298330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4a796bb7-0522-4e75-9117-cae0e898270d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:12.306356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1218/2423] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:12.310686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce6472-fa8b-40f8-b199-ed5e921c99de None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:12.320448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edce6472-fa8b-40f8-b199-ed5e921c99de None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:12.325048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:12.325709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.325992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.326268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.327010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:12.327346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:12.328974 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:12.340645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1206/2424] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:12.345229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a796bb7-0522-4e75-9117-cae0e898270d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:12.354457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a796bb7-0522-4e75-9117-cae0e898270d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:12.359017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6daa171-b9a4-4188-808a-d1b821fc5e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:12.359693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daa171-b9a4-4188-808a-d1b821fc5e86 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.359972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daa171-b9a4-4188-808a-d1b821fc5e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.360245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6daa171-b9a4-4188-808a-d1b821fc5e86 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.360999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6daa171-b9a4-4188-808a-d1b821fc5e86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:12.363045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6daa171-b9a4-4188-808a-d1b821fc5e86 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:12.373186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:12.393042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1219/2425] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:12 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:12.397324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:12.407896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7af55bf-e003-43d8-9abd-93fd86186c25 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:12.453102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:12.468220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed84060-937e-43c7-aeef-67669ba94a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:12.468823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed84060-937e-43c7-aeef-67669ba94a7d admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.469105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed84060-937e-43c7-aeef-67669ba94a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.469428 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed84060-937e-43c7-aeef-67669ba94a7d admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.470138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed84060-937e-43c7-aeef-67669ba94a7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:12.472127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed84060-937e-43c7-aeef-67669ba94a7d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:12.493809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1207/2426] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:12 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:12.498036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6daa171-b9a4-4188-808a-d1b821fc5e86 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:12.508754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6daa171-b9a4-4188-808a-d1b821fc5e86 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:12.513513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37499ff-b259-4b67-97cf-1cd4f78189c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:12.514191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37499ff-b259-4b67-97cf-1cd4f78189c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.514497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37499ff-b259-4b67-97cf-1cd4f78189c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.514770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37499ff-b259-4b67-97cf-1cd4f78189c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.515542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37499ff-b259-4b67-97cf-1cd4f78189c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:12.517507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37499ff-b259-4b67-97cf-1cd4f78189c9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:12.527714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1220/2427] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:12.532001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed84060-937e-43c7-aeef-67669ba94a7d admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:12.542177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed84060-937e-43c7-aeef-67669ba94a7d admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:12.546998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8133cf-5d6c-4ed6-a09a-9387925aa0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:12.547696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8133cf-5d6c-4ed6-a09a-9387925aa0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.548015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8133cf-5d6c-4ed6-a09a-9387925aa0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.548256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8133cf-5d6c-4ed6-a09a-9387925aa0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.549022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8133cf-5d6c-4ed6-a09a-9387925aa0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:12.550607 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8133cf-5d6c-4ed6-a09a-9387925aa0b7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:12.562425 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1208/2428] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:12.566857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37499ff-b259-4b67-97cf-1cd4f78189c9 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:12.578307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37499ff-b259-4b67-97cf-1cd4f78189c9 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:12.583847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:12.584770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.585057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.585331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] PATH_INFO: `/v3/projects/f730af25ff034a338304e09f55a9f596/users/eea7c4e3e9844a3eb6e7508d9016f52b/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.601950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f730af25ff034a338304e09f55a9f596, user_id=eea7c4e3e9844a3eb6e7508d9016f52b, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:12.604889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:12.620759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1221/2429] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:12 2022] PUT /identity/v3/projects/f730af25ff034a338304e09f55a9f596/users/eea7c4e3e9844a3eb6e7508d9016f52b/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:12.631229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505302-9da9-4dd6-ab5a-1da7be977a4c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.631780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505302-9da9-4dd6-ab5a-1da7be977a4c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.632273 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82505302-9da9-4dd6-ab5a-1da7be977a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.668924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-82505302-9da9-4dd6-ab5a-1da7be977a4c None None] MFA Rules not processed for user `eea7c4e3e9844a3eb6e7508d9016f52b`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:12.670182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-82505302-9da9-4dd6-ab5a-1da7be977a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:12.734072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1209/2430] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:12.745594 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:12.747323 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:12.826809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebb9e61-c0a1-4c65-9423-a41ccae6969b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:12.833102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fa8dc-d290-43f1-8271-51bc0327d83f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:12.833797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa8dc-d290-43f1-8271-51bc0327d83f service nova] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:12.834081 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa8dc-d290-43f1-8271-51bc0327d83f service nova] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:12.834354 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fa8dc-d290-43f1-8271-51bc0327d83f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:12.842916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fa8dc-d290-43f1-8271-51bc0327d83f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:12.844629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fa8dc-d290-43f1-8271-51bc0327d83f service nova] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:12.853586 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1222/2431] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:20:13.063473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505302-9da9-4dd6-ab5a-1da7be977a4c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:13.127370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82505302-9da9-4dd6-ab5a-1da7be977a4c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:13.132035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:13.132916 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:13.133190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:13.133459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] PATH_INFO: `/v3/users/eea7c4e3e9844a3eb6e7508d9016f52b` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:13.134647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea7c4e3e9844a3eb6e7508d9016f52b)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:13.136701 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:13.189874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea7c4e3e9844a3eb6e7508d9016f52b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:13.190535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea7c4e3e9844a3eb6e7508d9016f52b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:13.191141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea7c4e3e9844a3eb6e7508d9016f52b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:13.200962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea7c4e3e9844a3eb6e7508d9016f52b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:13.203399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:13.210523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea7c4e3e9844a3eb6e7508d9016f52b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:13.228999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1210/2432] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:13 2022] DELETE /identity/v3/users/eea7c4e3e9844a3eb6e7508d9016f52b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:13.233698 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257630a-6e38-4c94-8f2a-2bfea50594e2 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:13.233993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257630a-6e38-4c94-8f2a-2bfea50594e2 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:13.234271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b257630a-6e38-4c94-8f2a-2bfea50594e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:13.251754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b257630a-6e38-4c94-8f2a-2bfea50594e2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:13.253210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b257630a-6e38-4c94-8f2a-2bfea50594e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:13.321093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1223/2433] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:13.330906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:13.332458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:13.428070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b79a62-6794-4cf7-8be0-8ac0cdd54416 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:13.432125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc5b691-4a0f-4a66-821c-a9205a28d5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:13.432795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5b691-4a0f-4a66-821c-a9205a28d5be service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:13.433080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5b691-4a0f-4a66-821c-a9205a28d5be service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:13.433358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc5b691-4a0f-4a66-821c-a9205a28d5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:13.445910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5b691-4a0f-4a66-821c-a9205a28d5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:13.447410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc5b691-4a0f-4a66-821c-a9205a28d5be service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:13.458331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1211/2434] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:13.931213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257630a-6e38-4c94-8f2a-2bfea50594e2 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:13.932870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b257630a-6e38-4c94-8f2a-2bfea50594e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:13.945623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b257630a-6e38-4c94-8f2a-2bfea50594e2 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:13.953978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:13.957414 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:13.957702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:13.957982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] PATH_INFO: `/v3/projects/f730af25ff034a338304e09f55a9f596` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:13.960077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f730af25ff034a338304e09f55a9f596)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:13.962272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.008129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:14.020511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f730af25ff034a338304e09f55a9f596', 'request_id': 'req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:14.028036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f730af25ff034a338304e09f55a9f596', 'request_id': 'req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:14.034821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f730af25ff034a338304e09f55a9f596', 'request_id': 'req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:14.052419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] The token cache is being invalidate because project f730af25ff034a338304e09f55a9f596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:14.052743 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:14.053649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfa504d4-ef65-4755-8f6a-fc9abbd234dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:14.055913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1224/2435] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:13 2022] DELETE /identity/v3/projects/f730af25ff034a338304e09f55a9f596 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:14.060654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e62c-ca13-43cb-a28f-902cbfadb29e None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.061496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e62c-ca13-43cb-a28f-902cbfadb29e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.061936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1880e62c-ca13-43cb-a28f-902cbfadb29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.083882 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-1880e62c-ca13-43cb-a28f-902cbfadb29e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:14.086434 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1880e62c-ca13-43cb-a28f-902cbfadb29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:14.163144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1212/2436] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:14.171972 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1192-0aa2-41fb-8954-67c4e375213a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.172258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1192-0aa2-41fb-8954-67c4e375213a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.172688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603a1192-0aa2-41fb-8954-67c4e375213a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.193724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-603a1192-0aa2-41fb-8954-67c4e375213a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:14.195106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-603a1192-0aa2-41fb-8954-67c4e375213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:14.208629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1225/2437] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:14.214063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880e62c-ca13-43cb-a28f-902cbfadb29e None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.225845 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1880e62c-ca13-43cb-a28f-902cbfadb29e None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.231124 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.231886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.232174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.232451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.233192 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:14.233504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.235130 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.247616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1213/2438] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:14.252122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a1192-0aa2-41fb-8954-67c4e375213a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.261675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603a1192-0aa2-41fb-8954-67c4e375213a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.267245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59198-9df5-420b-927d-c37b67eade02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.267927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59198-9df5-420b-927d-c37b67eade02 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.268209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59198-9df5-420b-927d-c37b67eade02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.268474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f59198-9df5-420b-927d-c37b67eade02 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.269176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f59198-9df5-420b-927d-c37b67eade02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.271086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f59198-9df5-420b-927d-c37b67eade02 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.299378 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1226/2439] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:14 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:14.303191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.313545 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69764b6c-55ba-4d12-b8bd-0d3b63580e49 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.375982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac9066-5a41-4ff8-b0db-e23c10639d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.376672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac9066-5a41-4ff8-b0db-e23c10639d72 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.376954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac9066-5a41-4ff8-b0db-e23c10639d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.377224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dac9066-5a41-4ff8-b0db-e23c10639d72 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.377922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac9066-5a41-4ff8-b0db-e23c10639d72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.379971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dac9066-5a41-4ff8-b0db-e23c10639d72 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.389272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:14.407799 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1214/2440] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:14 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:14.409222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59198-9df5-420b-927d-c37b67eade02 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.423721 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f59198-9df5-420b-927d-c37b67eade02 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.430361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01af14-924c-41ac-9f6e-69f1cd6e447b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.431042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01af14-924c-41ac-9f6e-69f1cd6e447b admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.431346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01af14-924c-41ac-9f6e-69f1cd6e447b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.431677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01af14-924c-41ac-9f6e-69f1cd6e447b admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.432508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01af14-924c-41ac-9f6e-69f1cd6e447b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.434116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01af14-924c-41ac-9f6e-69f1cd6e447b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.445349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1227/2441] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:14.449477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac9066-5a41-4ff8-b0db-e23c10639d72 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.459781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dac9066-5a41-4ff8-b0db-e23c10639d72 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.464559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900eb59d-b66f-4eaa-9728-d74bb3323516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.465233 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eb59d-b66f-4eaa-9728-d74bb3323516 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.465516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eb59d-b66f-4eaa-9728-d74bb3323516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.465790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900eb59d-b66f-4eaa-9728-d74bb3323516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.466541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900eb59d-b66f-4eaa-9728-d74bb3323516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.468229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900eb59d-b66f-4eaa-9728-d74bb3323516 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.479016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1215/2442] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:14.483538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01af14-924c-41ac-9f6e-69f1cd6e447b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.494100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01af14-924c-41ac-9f6e-69f1cd6e447b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.499186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f2eb6-c09d-4fcb-b759-60df4e560b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.500053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f2eb6-c09d-4fcb-b759-60df4e560b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.500337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f2eb6-c09d-4fcb-b759-60df4e560b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.500610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f2eb6-c09d-4fcb-b759-60df4e560b97 admin admin] PATH_INFO: `/v3/projects/cf825fc687a14a538d9200b528ffe982/users/f79af114e46d445eba88b42fe2e57327/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.516268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f2eb6-c09d-4fcb-b759-60df4e560b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf825fc687a14a538d9200b528ffe982, user_id=f79af114e46d445eba88b42fe2e57327, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.519310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6f2eb6-c09d-4fcb-b759-60df4e560b97 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.545358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1228/2443] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:14 2022] PUT /identity/v3/projects/cf825fc687a14a538d9200b528ffe982/users/f79af114e46d445eba88b42fe2e57327/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:14.547370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b11d-5be6-4b1e-900d-1423ff556106 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.547691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b11d-5be6-4b1e-900d-1423ff556106 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.548248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732b11d-5be6-4b1e-900d-1423ff556106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.578072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-2732b11d-5be6-4b1e-900d-1423ff556106 None None] MFA Rules not processed for user `f79af114e46d445eba88b42fe2e57327`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:14.579411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2732b11d-5be6-4b1e-900d-1423ff556106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:14.659224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1216/2444] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:14.663820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f2eb6-c09d-4fcb-b759-60df4e560b97 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.738773 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f2eb6-c09d-4fcb-b759-60df4e560b97 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.743850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a3315-8079-4dec-88da-e9ef672b53a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.744514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a3315-8079-4dec-88da-e9ef672b53a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.744800 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a3315-8079-4dec-88da-e9ef672b53a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.745078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1a3315-8079-4dec-88da-e9ef672b53a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.745797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a3315-8079-4dec-88da-e9ef672b53a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.747776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1a3315-8079-4dec-88da-e9ef672b53a3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.782921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1229/2445] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:14 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:14.786955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732b11d-5be6-4b1e-900d-1423ff556106 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.797798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732b11d-5be6-4b1e-900d-1423ff556106 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.855770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea28e-8d77-4d0d-ae7d-b123ad02562e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.856360 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea28e-8d77-4d0d-ae7d-b123ad02562e admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.856651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea28e-8d77-4d0d-ae7d-b123ad02562e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.856927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ea28e-8d77-4d0d-ae7d-b123ad02562e admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.857626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ea28e-8d77-4d0d-ae7d-b123ad02562e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.859618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ea28e-8d77-4d0d-ae7d-b123ad02562e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.881442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1217/2446] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:14 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:14.885737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a3315-8079-4dec-88da-e9ef672b53a3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.896776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1a3315-8079-4dec-88da-e9ef672b53a3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.902138 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5159d1-2b12-4f31-8259-be84e2e87884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.902771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5159d1-2b12-4f31-8259-be84e2e87884 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.903051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5159d1-2b12-4f31-8259-be84e2e87884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.903356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5159d1-2b12-4f31-8259-be84e2e87884 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.904102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5159d1-2b12-4f31-8259-be84e2e87884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.905703 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5159d1-2b12-4f31-8259-be84e2e87884 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.916933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1230/2447] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:14.921095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea28e-8d77-4d0d-ae7d-b123ad02562e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.931858 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ea28e-8d77-4d0d-ae7d-b123ad02562e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:14.937385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179047-4e4c-4df4-abfd-413c7cd4efdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:14.938270 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179047-4e4c-4df4-abfd-413c7cd4efdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:14.938560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179047-4e4c-4df4-abfd-413c7cd4efdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:14.938833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179047-4e4c-4df4-abfd-413c7cd4efdc admin admin] PATH_INFO: `/v3/projects/45165529152549ada8201787ddc7e88c/users/0d52133401e249159103ae4d163e54be/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:14.955187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b179047-4e4c-4df4-abfd-413c7cd4efdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=45165529152549ada8201787ddc7e88c, user_id=0d52133401e249159103ae4d163e54be, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:14.957912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b179047-4e4c-4df4-abfd-413c7cd4efdc admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:14.973869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1218/2448] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:14 2022] PUT /identity/v3/projects/45165529152549ada8201787ddc7e88c/users/0d52133401e249159103ae4d163e54be/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:14.977969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5159d1-2b12-4f31-8259-be84e2e87884 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:14.988900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5159d1-2b12-4f31-8259-be84e2e87884 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:15.022402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:15.048724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28124fdd-2642-4efc-a846-7199f8f1b39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:15.049394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28124fdd-2642-4efc-a846-7199f8f1b39f admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:15.049672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28124fdd-2642-4efc-a846-7199f8f1b39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:15.049947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28124fdd-2642-4efc-a846-7199f8f1b39f admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:15.050677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28124fdd-2642-4efc-a846-7199f8f1b39f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:15.052334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28124fdd-2642-4efc-a846-7199f8f1b39f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:15.061969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1231/2449] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:15.066503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179047-4e4c-4df4-abfd-413c7cd4efdc admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:15.077164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179047-4e4c-4df4-abfd-413c7cd4efdc admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:15.082025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:15.083022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:15.083339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:15.083614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] PATH_INFO: `/v3/projects/45165529152549ada8201787ddc7e88c/users/0d52133401e249159103ae4d163e54be/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:15.085804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45165529152549ada8201787ddc7e88c, user_id=0d52133401e249159103ae4d163e54be, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:15.088531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:15.099971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1219/2450] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:15 2022] PUT /identity/v3/projects/45165529152549ada8201787ddc7e88c/users/0d52133401e249159103ae4d163e54be/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:15.110324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844f4b6-d56a-4337-9a27-f3135fafa208 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:15.110554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844f4b6-d56a-4337-9a27-f3135fafa208 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:15.110841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844f4b6-d56a-4337-9a27-f3135fafa208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:15.140178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5844f4b6-d56a-4337-9a27-f3135fafa208 None None] MFA Rules not processed for user `0d52133401e249159103ae4d163e54be`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:15.141403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5844f4b6-d56a-4337-9a27-f3135fafa208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:15.223987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1232/2451] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:15.235687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:15.237474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:15.265507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:15.297041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faea92e-3a39-4827-98b0-7816946c58e8 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:15.301008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3548ede-e54f-400f-b357-720f84c7e281 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:15.301649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3548ede-e54f-400f-b357-720f84c7e281 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:15.301925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3548ede-e54f-400f-b357-720f84c7e281 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:15.302197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3548ede-e54f-400f-b357-720f84c7e281 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:15.311640 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3548ede-e54f-400f-b357-720f84c7e281 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:15.313085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3548ede-e54f-400f-b357-720f84c7e281 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:15.322998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1220/2452] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:15 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:20:15.769303 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844f4b6-d56a-4337-9a27-f3135fafa208 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:15.840613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844f4b6-d56a-4337-9a27-f3135fafa208 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:15.845346 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:15.846248 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:15.846531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:15.846807 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] PATH_INFO: `/v3/users/f79af114e46d445eba88b42fe2e57327` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:15.848028 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79af114e46d445eba88b42fe2e57327)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:15.849971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:15.900388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79af114e46d445eba88b42fe2e57327'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:15.907200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79af114e46d445eba88b42fe2e57327'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:15.907952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79af114e46d445eba88b42fe2e57327'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:15.910361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:15.918612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79af114e46d445eba88b42fe2e57327'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:15.919945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79af114e46d445eba88b42fe2e57327'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:15.937753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1233/2453] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:15 2022] DELETE /identity/v3/users/f79af114e46d445eba88b42fe2e57327 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:15.942580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f366f48-5c5a-415d-bb19-37bb8b3656de None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:15.942864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f366f48-5c5a-415d-bb19-37bb8b3656de None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:15.943141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f366f48-5c5a-415d-bb19-37bb8b3656de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:15.960244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-9f366f48-5c5a-415d-bb19-37bb8b3656de None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:15.961431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9f366f48-5c5a-415d-bb19-37bb8b3656de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:16.020405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1221/2454] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:16.030350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:16.032140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:16.088007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63306603-cffa-486b-a21a-93f2d9188f50 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:16.092086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5a83-ccfc-4131-9144-c525ed6799c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:16.092733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5a83-ccfc-4131-9144-c525ed6799c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:16.093033 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5a83-ccfc-4131-9144-c525ed6799c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:16.093298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5a83-ccfc-4131-9144-c525ed6799c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:16.104184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5a83-ccfc-4131-9144-c525ed6799c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:16.105621 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5a83-ccfc-4131-9144-c525ed6799c4 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:16.115026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1234/2455] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:16.404781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f366f48-5c5a-415d-bb19-37bb8b3656de None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:16.406350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9f366f48-5c5a-415d-bb19-37bb8b3656de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:16.418933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f366f48-5c5a-415d-bb19-37bb8b3656de None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:16.424631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:16.425618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:16.427547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:16.428005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] PATH_INFO: `/v3/projects/cf825fc687a14a538d9200b528ffe982` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:16.429478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf825fc687a14a538d9200b528ffe982)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:16.431932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:16.481242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf825fc687a14a538d9200b528ffe982', 'request_id': 'req-8734d219-cdc0-425a-9fc2-98060bc86d01'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.491250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf825fc687a14a538d9200b528ffe982', 'request_id': 'req-8734d219-cdc0-425a-9fc2-98060bc86d01'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.496785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf825fc687a14a538d9200b528ffe982', 'request_id': 'req-8734d219-cdc0-425a-9fc2-98060bc86d01'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.507462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] The token cache is being invalidate because project cf825fc687a14a538d9200b528ffe982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:16.507746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.508348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.509874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1222/2456] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:16 2022] DELETE /identity/v3/projects/cf825fc687a14a538d9200b528ffe982 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:16.513958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5a83-ccfc-4131-9144-c525ed6799c4 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:16.524117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5a83-ccfc-4131-9144-c525ed6799c4 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:16.580833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:16.581698 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:16.581977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:16.582241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] PATH_INFO: `/v3/users/0d52133401e249159103ae4d163e54be` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:16.583501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d52133401e249159103ae4d163e54be)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:16.585736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:16.647951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d52133401e249159103ae4d163e54be'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.654661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d52133401e249159103ae4d163e54be'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.655344 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d52133401e249159103ae4d163e54be'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.664627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d52133401e249159103ae4d163e54be'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.665314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4af6cfb1-5a91-45fe-855a-ea0aaea0368f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d52133401e249159103ae4d163e54be'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:16.679385 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:16.685380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1235/2457] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:16 2022] DELETE /identity/v3/users/0d52133401e249159103ae4d163e54be => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:16.955552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:16.957164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:16.970268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8734d219-cdc0-425a-9fc2-98060bc86d01 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.019146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:17.031250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.032094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.032380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.032653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] PATH_INFO: `/v3/projects/45165529152549ada8201787ddc7e88c` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.033835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] RBAC: Authorizing `identity:delete_project(project_id=45165529152549ada8201787ddc7e88c)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.035791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.082926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45165529152549ada8201787ddc7e88c', 'request_id': 'req-bb8b9084-39e1-4f2d-b2ee-929a5135946b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:17.092747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45165529152549ada8201787ddc7e88c', 'request_id': 'req-bb8b9084-39e1-4f2d-b2ee-929a5135946b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:17.098957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45165529152549ada8201787ddc7e88c', 'request_id': 'req-bb8b9084-39e1-4f2d-b2ee-929a5135946b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:17.115627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] The token cache is being invalidate because project 45165529152549ada8201787ddc7e88c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:17.115926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:17.116661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bb8b9084-39e1-4f2d-b2ee-929a5135946b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:17.118327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1223/2458] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:16 2022] DELETE /identity/v3/projects/45165529152549ada8201787ddc7e88c => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:17.124545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bd1c-ca74-4de3-b5de-1a858a85906c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.125012 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bd1c-ca74-4de3-b5de-1a858a85906c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.125114 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9763bd1c-ca74-4de3-b5de-1a858a85906c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.144174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-9763bd1c-ca74-4de3-b5de-1a858a85906c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:17.145394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9763bd1c-ca74-4de3-b5de-1a858a85906c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:17.206097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1236/2459] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:17.215687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f12e5f-a5e5-4a22-9053-b67c2619cd4a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.216116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f12e5f-a5e5-4a22-9053-b67c2619cd4a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.216494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f12e5f-a5e5-4a22-9053-b67c2619cd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.237427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-82f12e5f-a5e5-4a22-9053-b67c2619cd4a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:17.238647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-82f12e5f-a5e5-4a22-9053-b67c2619cd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:17.247912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1224/2460] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:17.252366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763bd1c-ca74-4de3-b5de-1a858a85906c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.262222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9763bd1c-ca74-4de3-b5de-1a858a85906c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.267684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.268349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.268627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.268905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.269673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:17.269993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.271672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.283668 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1237/2461] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:20:17.288119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f12e5f-a5e5-4a22-9053-b67c2619cd4a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.298365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f12e5f-a5e5-4a22-9053-b67c2619cd4a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.303363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7b1e-e0d9-4a18-865c-ac8f83b4bebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.304021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7b1e-e0d9-4a18-865c-ac8f83b4bebb admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.304305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7b1e-e0d9-4a18-865c-ac8f83b4bebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.304580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069a7b1e-e0d9-4a18-865c-ac8f83b4bebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.305279 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a7b1e-e0d9-4a18-865c-ac8f83b4bebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.307349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069a7b1e-e0d9-4a18-865c-ac8f83b4bebb admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.335768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1225/2462] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:17 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:17.340124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.350268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60731557-2324-4209-8b03-907d1b5a31f0 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.410349 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce881874-28bf-4471-b2cd-5ddacbb375db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.411039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881874-28bf-4471-b2cd-5ddacbb375db admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.411335 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881874-28bf-4471-b2cd-5ddacbb375db admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.411610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce881874-28bf-4471-b2cd-5ddacbb375db admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.412307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce881874-28bf-4471-b2cd-5ddacbb375db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.414257 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce881874-28bf-4471-b2cd-5ddacbb375db admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.435299 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1238/2463] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:17 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:17.439747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7b1e-e0d9-4a18-865c-ac8f83b4bebb admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.450566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069a7b1e-e0d9-4a18-865c-ac8f83b4bebb admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.456709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77850a-3aff-4d57-8f53-9517d776d589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.457399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77850a-3aff-4d57-8f53-9517d776d589 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.457679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77850a-3aff-4d57-8f53-9517d776d589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.457945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77850a-3aff-4d57-8f53-9517d776d589 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.458673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77850a-3aff-4d57-8f53-9517d776d589 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.460311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77850a-3aff-4d57-8f53-9517d776d589 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.471348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1226/2464] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:17.475226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce881874-28bf-4471-b2cd-5ddacbb375db admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.485958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce881874-28bf-4471-b2cd-5ddacbb375db admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.491178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742485b7-97e6-4a0b-9976-21962b6df8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.491987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742485b7-97e6-4a0b-9976-21962b6df8a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.492296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742485b7-97e6-4a0b-9976-21962b6df8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.492569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742485b7-97e6-4a0b-9976-21962b6df8a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.493307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742485b7-97e6-4a0b-9976-21962b6df8a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.494958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742485b7-97e6-4a0b-9976-21962b6df8a6 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.505457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1239/2465] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:17.509745 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77850a-3aff-4d57-8f53-9517d776d589 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.520399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77850a-3aff-4d57-8f53-9517d776d589 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.525692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cf77b-d13d-4719-b2d1-72002b1539f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.526623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cf77b-d13d-4719-b2d1-72002b1539f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.526923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cf77b-d13d-4719-b2d1-72002b1539f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.527199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471cf77b-d13d-4719-b2d1-72002b1539f1 admin admin] PATH_INFO: `/v3/projects/056cac4d01e742d8b2f5e3904c7dca96/users/c65fbc471b1e492ab83a8f6369c3722d/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.542776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cf77b-d13d-4719-b2d1-72002b1539f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056cac4d01e742d8b2f5e3904c7dca96, user_id=c65fbc471b1e492ab83a8f6369c3722d, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.545528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471cf77b-d13d-4719-b2d1-72002b1539f1 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.560800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1227/2466] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:17 2022] PUT /identity/v3/projects/056cac4d01e742d8b2f5e3904c7dca96/users/c65fbc471b1e492ab83a8f6369c3722d/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:17.572982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eb9ff-3a86-434d-bc9a-03914458e8e2 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.573260 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eb9ff-3a86-434d-bc9a-03914458e8e2 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.573539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3eb9ff-3a86-434d-bc9a-03914458e8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.600395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4b3eb9ff-3a86-434d-bc9a-03914458e8e2 None None] MFA Rules not processed for user `c65fbc471b1e492ab83a8f6369c3722d`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:17.601608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4b3eb9ff-3a86-434d-bc9a-03914458e8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:17.665252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:17.680831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1240/2467] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:17 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:17.682570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cf77b-d13d-4719-b2d1-72002b1539f1 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.747617 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471cf77b-d13d-4719-b2d1-72002b1539f1 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.752673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c756f99-7daf-43ad-b382-9ae64d2f265c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.753330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c756f99-7daf-43ad-b382-9ae64d2f265c admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.753614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c756f99-7daf-43ad-b382-9ae64d2f265c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.753884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c756f99-7daf-43ad-b382-9ae64d2f265c admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.754576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c756f99-7daf-43ad-b382-9ae64d2f265c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.756535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c756f99-7daf-43ad-b382-9ae64d2f265c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.783639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1228/2468] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:17 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:17.788109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3eb9ff-3a86-434d-bc9a-03914458e8e2 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.799361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3eb9ff-3a86-434d-bc9a-03914458e8e2 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.866534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab7077-9053-4778-9064-ea1d559ddfcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.867202 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab7077-9053-4778-9064-ea1d559ddfcb admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.867524 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab7077-9053-4778-9064-ea1d559ddfcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.867802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab7077-9053-4778-9064-ea1d559ddfcb admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.868513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab7077-9053-4778-9064-ea1d559ddfcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.870472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab7077-9053-4778-9064-ea1d559ddfcb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.895620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1241/2469] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:17 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:17.899749 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c756f99-7daf-43ad-b382-9ae64d2f265c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.905889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:17.913493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c756f99-7daf-43ad-b382-9ae64d2f265c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.919523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf370aa-6260-45e3-b8db-c397fcb7a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.920149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf370aa-6260-45e3-b8db-c397fcb7a4ac admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.920436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf370aa-6260-45e3-b8db-c397fcb7a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.920747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf370aa-6260-45e3-b8db-c397fcb7a4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.921481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf370aa-6260-45e3-b8db-c397fcb7a4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.923750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf370aa-6260-45e3-b8db-c397fcb7a4ac admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.939389 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1229/2470] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:17.943771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab7077-9053-4778-9064-ea1d559ddfcb admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.955497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab7077-9053-4778-9064-ea1d559ddfcb admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.960427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a1faa9-ec42-4b5e-bfe0-ded5a7ceab80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.961101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1faa9-ec42-4b5e-bfe0-ded5a7ceab80 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.961413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1faa9-ec42-4b5e-bfe0-ded5a7ceab80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.961696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a1faa9-ec42-4b5e-bfe0-ded5a7ceab80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:17.962465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1faa9-ec42-4b5e-bfe0-ded5a7ceab80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:17.964123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a1faa9-ec42-4b5e-bfe0-ded5a7ceab80 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:17.974263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1242/2471] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:17.978451 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf370aa-6260-45e3-b8db-c397fcb7a4ac admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:17.988830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf370aa-6260-45e3-b8db-c397fcb7a4ac admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:17.993541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f890d-0227-4216-a91f-1ed7a9c50733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:17.994515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f890d-0227-4216-a91f-1ed7a9c50733 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:17.994800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f890d-0227-4216-a91f-1ed7a9c50733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:17.995077 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f890d-0227-4216-a91f-1ed7a9c50733 admin admin] PATH_INFO: `/v3/projects/552610590b1446d6a692ae02621f6938/users/8b3f675248074dce8d5a141d9b2baec4/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.010669 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f890d-0227-4216-a91f-1ed7a9c50733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=552610590b1446d6a692ae02621f6938, user_id=8b3f675248074dce8d5a141d9b2baec4, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.013508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f890d-0227-4216-a91f-1ed7a9c50733 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.027164 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1230/2472] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:17 2022] PUT /identity/v3/projects/552610590b1446d6a692ae02621f6938/users/8b3f675248074dce8d5a141d9b2baec4/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:18.037416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c410c-afe7-4771-80e2-049db3fd0954 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.037732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c410c-afe7-4771-80e2-049db3fd0954 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.038008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0c410c-afe7-4771-80e2-049db3fd0954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.065459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-db0c410c-afe7-4771-80e2-049db3fd0954 None None] MFA Rules not processed for user `8b3f675248074dce8d5a141d9b2baec4`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:18.066600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-db0c410c-afe7-4771-80e2-049db3fd0954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:18.139310 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1243/2473] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:18.140727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f890d-0227-4216-a91f-1ed7a9c50733 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:18.201289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f890d-0227-4216-a91f-1ed7a9c50733 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:18.207025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab956f0-b2e1-4d85-a27e-ddb1935dea5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:18.207716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab956f0-b2e1-4d85-a27e-ddb1935dea5f admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.208016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab956f0-b2e1-4d85-a27e-ddb1935dea5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.208274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab956f0-b2e1-4d85-a27e-ddb1935dea5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.208957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab956f0-b2e1-4d85-a27e-ddb1935dea5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.210864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab956f0-b2e1-4d85-a27e-ddb1935dea5f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.238563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1231/2474] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:18 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:18.242788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c410c-afe7-4771-80e2-049db3fd0954 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:18.253505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0c410c-afe7-4771-80e2-049db3fd0954 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:18.312372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aabf7-62e4-4730-9005-40c2e8a8cb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:18.313049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aabf7-62e4-4730-9005-40c2e8a8cb5d admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.313332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aabf7-62e4-4730-9005-40c2e8a8cb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.313599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0aabf7-62e4-4730-9005-40c2e8a8cb5d admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.314286 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0aabf7-62e4-4730-9005-40c2e8a8cb5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.316911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0aabf7-62e4-4730-9005-40c2e8a8cb5d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.325229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:18.342629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1244/2475] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:18 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:18.343718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab956f0-b2e1-4d85-a27e-ddb1935dea5f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:18.354407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab956f0-b2e1-4d85-a27e-ddb1935dea5f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:18.359034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1a53b-b510-4f49-a182-5c5fc7b89614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:18.359735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1a53b-b510-4f49-a182-5c5fc7b89614 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.360710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1a53b-b510-4f49-a182-5c5fc7b89614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.360710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1a53b-b510-4f49-a182-5c5fc7b89614 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.361039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1a53b-b510-4f49-a182-5c5fc7b89614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.362653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b1a53b-b510-4f49-a182-5c5fc7b89614 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.373376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1232/2476] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:18.377679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aabf7-62e4-4730-9005-40c2e8a8cb5d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:18.388275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0aabf7-62e4-4730-9005-40c2e8a8cb5d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:18.393904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a1412-4a6e-4190-a1e0-7372b10c3532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:18.394799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1412-4a6e-4190-a1e0-7372b10c3532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.395079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1412-4a6e-4190-a1e0-7372b10c3532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.395389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1412-4a6e-4190-a1e0-7372b10c3532 admin admin] PATH_INFO: `/v3/projects/007a174761b349d7b258ef4089bdb456/users/638864cc7b5e410aadf89dfd81332ab9/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.410134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a1412-4a6e-4190-a1e0-7372b10c3532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=007a174761b349d7b258ef4089bdb456, user_id=638864cc7b5e410aadf89dfd81332ab9, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.412837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a1412-4a6e-4190-a1e0-7372b10c3532 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.427511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1245/2477] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:18 2022] PUT /identity/v3/projects/007a174761b349d7b258ef4089bdb456/users/638864cc7b5e410aadf89dfd81332ab9/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:18.431676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1a53b-b510-4f49-a182-5c5fc7b89614 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:18.441820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b1a53b-b510-4f49-a182-5c5fc7b89614 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:18.497241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d80981-ef45-417e-ac10-b1856c1b1503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:18.497898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80981-ef45-417e-ac10-b1856c1b1503 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.498206 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80981-ef45-417e-ac10-b1856c1b1503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.498477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d80981-ef45-417e-ac10-b1856c1b1503 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.499204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d80981-ef45-417e-ac10-b1856c1b1503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.500824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d80981-ef45-417e-ac10-b1856c1b1503 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.510767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1233/2478] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:18.514999 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a1412-4a6e-4190-a1e0-7372b10c3532 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:18.525557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a1412-4a6e-4190-a1e0-7372b10c3532 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:18.531851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:18.532783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.533018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.533326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] PATH_INFO: `/v3/projects/007a174761b349d7b258ef4089bdb456/users/638864cc7b5e410aadf89dfd81332ab9/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.535489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=007a174761b349d7b258ef4089bdb456, user_id=638864cc7b5e410aadf89dfd81332ab9, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.538110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.555702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1246/2479] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:18 2022] PUT /identity/v3/projects/007a174761b349d7b258ef4089bdb456/users/638864cc7b5e410aadf89dfd81332ab9/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:18.566885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be8e5f-7918-4218-ba47-a9fe1b643725 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.567250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be8e5f-7918-4218-ba47-a9fe1b643725 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.567495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be8e5f-7918-4218-ba47-a9fe1b643725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.601033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-24be8e5f-7918-4218-ba47-a9fe1b643725 None None] MFA Rules not processed for user `638864cc7b5e410aadf89dfd81332ab9`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:18.602240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-24be8e5f-7918-4218-ba47-a9fe1b643725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:18.637335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:18.693787 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1234/2480] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:18 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:18.708189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:18.710616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:18.778851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e453115-0958-4db3-9a03-92c7e7a94195 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:18.783345 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17804950-fdb3-4f9d-991d-8b821ecb1696 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:18.784003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17804950-fdb3-4f9d-991d-8b821ecb1696 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.784389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17804950-fdb3-4f9d-991d-8b821ecb1696 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.784721 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17804950-fdb3-4f9d-991d-8b821ecb1696 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.795373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17804950-fdb3-4f9d-991d-8b821ecb1696 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.802034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17804950-fdb3-4f9d-991d-8b821ecb1696 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.809363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1247/2481] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:18 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:18.857623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be8e5f-7918-4218-ba47-a9fe1b643725 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:18.859424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-24be8e5f-7918-4218-ba47-a9fe1b643725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:18.871743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be8e5f-7918-4218-ba47-a9fe1b643725 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:18.875780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f34e58e-326e-4818-9054-5879e9115070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:18.876442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34e58e-326e-4818-9054-5879e9115070 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:18.876727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34e58e-326e-4818-9054-5879e9115070 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:18.877022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34e58e-326e-4818-9054-5879e9115070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:18.935499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34e58e-326e-4818-9054-5879e9115070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:18.936960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f34e58e-326e-4818-9054-5879e9115070 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:18.945765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1235/2482] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:24.023794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17804950-fdb3-4f9d-991d-8b821ecb1696 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:24.088974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17804950-fdb3-4f9d-991d-8b821ecb1696 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:24.093602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:24.094467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:24.094741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:24.095009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] PATH_INFO: `/v3/users/c65fbc471b1e492ab83a8f6369c3722d` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:24.096215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c65fbc471b1e492ab83a8f6369c3722d)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:24.098134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:24.150759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c65fbc471b1e492ab83a8f6369c3722d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.157498 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c65fbc471b1e492ab83a8f6369c3722d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.158164 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c65fbc471b1e492ab83a8f6369c3722d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.160815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:24.168772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c65fbc471b1e492ab83a8f6369c3722d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.169578 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c65fbc471b1e492ab83a8f6369c3722d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.187569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1248/2483] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:24 2022] DELETE /identity/v3/users/c65fbc471b1e492ab83a8f6369c3722d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:24.192411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e0b39-3ad0-43d5-8644-5a045415c034 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:24.192908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e0b39-3ad0-43d5-8644-5a045415c034 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:24.192908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e0b39-3ad0-43d5-8644-5a045415c034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:24.212441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-c33e0b39-3ad0-43d5-8644-5a045415c034 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:24.213764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c33e0b39-3ad0-43d5-8644-5a045415c034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:24.277108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1236/2484] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:24.286938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:24.288365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:24.345459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c0991-0c8d-4012-ba1f-bc6d2e7b540b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:24.349393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c53af-af03-4de9-accd-5c9b2e4c8c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:24.350042 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c53af-af03-4de9-accd-5c9b2e4c8c65 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:24.350318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c53af-af03-4de9-accd-5c9b2e4c8c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:24.350590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c53af-af03-4de9-accd-5c9b2e4c8c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:24.360191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c53af-af03-4de9-accd-5c9b2e4c8c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:24.361649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c53af-af03-4de9-accd-5c9b2e4c8c65 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:24.371174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1249/2485] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:24.644553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e0b39-3ad0-43d5-8644-5a045415c034 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:24.646020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c33e0b39-3ad0-43d5-8644-5a045415c034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:24.664729 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e0b39-3ad0-43d5-8644-5a045415c034 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:24.674365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:24.675324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:24.676224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:24.676527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] PATH_INFO: `/v3/projects/056cac4d01e742d8b2f5e3904c7dca96` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:24.677957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=056cac4d01e742d8b2f5e3904c7dca96)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:24.679940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:24.729927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056cac4d01e742d8b2f5e3904c7dca96', 'request_id': 'req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.739117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056cac4d01e742d8b2f5e3904c7dca96', 'request_id': 'req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.740731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:24.746976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056cac4d01e742d8b2f5e3904c7dca96', 'request_id': 'req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.761863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] The token cache is being invalidate because project 056cac4d01e742d8b2f5e3904c7dca96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:24.762168 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.762861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.764404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1237/2486] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:24 2022] DELETE /identity/v3/projects/056cac4d01e742d8b2f5e3904c7dca96 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:24.768978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c53af-af03-4de9-accd-5c9b2e4c8c65 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:24.779769 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c53af-af03-4de9-accd-5c9b2e4c8c65 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:24.839324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:24.840230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:24.840510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:24.840785 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] PATH_INFO: `/v3/users/8b3f675248074dce8d5a141d9b2baec4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:24.841982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3f675248074dce8d5a141d9b2baec4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:24.843981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:24.896105 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3f675248074dce8d5a141d9b2baec4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.902969 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3f675248074dce8d5a141d9b2baec4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.903673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3f675248074dce8d5a141d9b2baec4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.912611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3f675248074dce8d5a141d9b2baec4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.913228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3f675248074dce8d5a141d9b2baec4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:24.927180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:24.933046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1250/2487] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:24 2022] DELETE /identity/v3/users/8b3f675248074dce8d5a141d9b2baec4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:25.223837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:25.225374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:25.240504 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f054a-99fa-42fd-86eb-bdb5d3727b22 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:25.299804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:25.300597 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:25.300881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:25.301154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] PATH_INFO: `/v3/projects/552610590b1446d6a692ae02621f6938` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:25.302358 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=552610590b1446d6a692ae02621f6938)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:25.304332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:25.352873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '552610590b1446d6a692ae02621f6938', 'request_id': 'req-7fc1ec89-25de-40f6-b26e-6842c299b4cc'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.361951 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '552610590b1446d6a692ae02621f6938', 'request_id': 'req-7fc1ec89-25de-40f6-b26e-6842c299b4cc'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.367703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '552610590b1446d6a692ae02621f6938', 'request_id': 'req-7fc1ec89-25de-40f6-b26e-6842c299b4cc'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.378807 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] The token cache is being invalidate because project 552610590b1446d6a692ae02621f6938 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:25.379097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.380038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.381340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1238/2488] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:25 2022] DELETE /identity/v3/projects/552610590b1446d6a692ae02621f6938 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:25.385347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:25.386940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:25.399820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6220ba00-8ef2-4b8a-8dd4-9bbe64235b3b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:25.458849 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:25.459776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:25.460061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:25.460386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] PATH_INFO: `/v3/users/638864cc7b5e410aadf89dfd81332ab9` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:25.461609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] RBAC: Authorizing `identity:delete_user(user_id=638864cc7b5e410aadf89dfd81332ab9)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:25.463568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:25.526232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '638864cc7b5e410aadf89dfd81332ab9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.533491 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '638864cc7b5e410aadf89dfd81332ab9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.534258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '638864cc7b5e410aadf89dfd81332ab9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.543420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '638864cc7b5e410aadf89dfd81332ab9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.544062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-d341b9a8-910e-4499-a1c7-81f2b62d6647 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '638864cc7b5e410aadf89dfd81332ab9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.561282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1251/2489] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:25 2022] DELETE /identity/v3/users/638864cc7b5e410aadf89dfd81332ab9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:25.838739 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:25.840863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:25.855818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc1ec89-25de-40f6-b26e-6842c299b4cc admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:25.915756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:25.916630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:25.916908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:25.917180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] PATH_INFO: `/v3/projects/007a174761b349d7b258ef4089bdb456` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:25.918406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] RBAC: Authorizing `identity:delete_project(project_id=007a174761b349d7b258ef4089bdb456)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:25.920365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:25.936918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:25.968589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '007a174761b349d7b258ef4089bdb456', 'request_id': 'req-503f0304-4346-454a-954c-9a6e7b7fb07c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.976292 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '007a174761b349d7b258ef4089bdb456', 'request_id': 'req-503f0304-4346-454a-954c-9a6e7b7fb07c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.983716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '007a174761b349d7b258ef4089bdb456', 'request_id': 'req-503f0304-4346-454a-954c-9a6e7b7fb07c'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.994791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] The token cache is being invalidate because project 007a174761b349d7b258ef4089bdb456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:25.995073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.995765 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-503f0304-4346-454a-954c-9a6e7b7fb07c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:25.997571 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1239/2490] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:25 2022] DELETE /identity/v3/projects/007a174761b349d7b258ef4089bdb456 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:26.003670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e94a-8edd-457d-a63f-d3e4e17d14ab None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.003960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e94a-8edd-457d-a63f-d3e4e17d14ab None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.004237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0e94a-8edd-457d-a63f-d3e4e17d14ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.022545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-4db0e94a-8edd-457d-a63f-d3e4e17d14ab None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:26.023754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4db0e94a-8edd-457d-a63f-d3e4e17d14ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:26.089008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1252/2491] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:26.097674 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd2f5-ca00-4bf4-90e5-84fedfd4f304 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.098038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd2f5-ca00-4bf4-90e5-84fedfd4f304 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.098383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163fd2f5-ca00-4bf4-90e5-84fedfd4f304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.118229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-163fd2f5-ca00-4bf4-90e5-84fedfd4f304 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:26.119569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-163fd2f5-ca00-4bf4-90e5-84fedfd4f304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:26.129622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1240/2492] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:26.134482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0e94a-8edd-457d-a63f-d3e4e17d14ab None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.145436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0e94a-8edd-457d-a63f-d3e4e17d14ab None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.150635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.151295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.151603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.151877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.152658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:26.152935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.154557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.168775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1253/2493] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:26.173842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fd2f5-ca00-4bf4-90e5-84fedfd4f304 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.183368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163fd2f5-ca00-4bf4-90e5-84fedfd4f304 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.190019 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31b3d6-7110-4f78-8638-0dd9d04407cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.190367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31b3d6-7110-4f78-8638-0dd9d04407cb admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.190649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31b3d6-7110-4f78-8638-0dd9d04407cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.190925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac31b3d6-7110-4f78-8638-0dd9d04407cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.191901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31b3d6-7110-4f78-8638-0dd9d04407cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.193869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac31b3d6-7110-4f78-8638-0dd9d04407cb admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.228267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1241/2494] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:26 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:26.232487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.241558 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:26.245449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63c401-804c-4a80-a5a7-e4d5099cae92 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.303149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2e2f-3bbb-4709-aae2-27953e616cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.303842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2e2f-3bbb-4709-aae2-27953e616cca admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.304122 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2e2f-3bbb-4709-aae2-27953e616cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.304395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c2e2f-3bbb-4709-aae2-27953e616cca admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.305108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c2e2f-3bbb-4709-aae2-27953e616cca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.307024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04c2e2f-3bbb-4709-aae2-27953e616cca admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.326815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1254/2495] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:26 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:26.331038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31b3d6-7110-4f78-8638-0dd9d04407cb admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.342160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac31b3d6-7110-4f78-8638-0dd9d04407cb admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.347087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5ebfa-bdcc-4e07-b17f-a6b93734715a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.347822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5ebfa-bdcc-4e07-b17f-a6b93734715a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.348086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5ebfa-bdcc-4e07-b17f-a6b93734715a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.348382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5ebfa-bdcc-4e07-b17f-a6b93734715a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.349141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5ebfa-bdcc-4e07-b17f-a6b93734715a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.350837 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5ebfa-bdcc-4e07-b17f-a6b93734715a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.368688 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1242/2496] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:26.372641 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2e2f-3bbb-4709-aae2-27953e616cca admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.382889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04c2e2f-3bbb-4709-aae2-27953e616cca admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.388683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a019f6d-b735-48d0-86dd-7c4f0e54c4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.389362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a019f6d-b735-48d0-86dd-7c4f0e54c4de admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.389642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a019f6d-b735-48d0-86dd-7c4f0e54c4de admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.389920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a019f6d-b735-48d0-86dd-7c4f0e54c4de admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.390720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a019f6d-b735-48d0-86dd-7c4f0e54c4de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.392774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a019f6d-b735-48d0-86dd-7c4f0e54c4de admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.403501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1255/2497] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:26.407832 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5ebfa-bdcc-4e07-b17f-a6b93734715a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.418909 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5ebfa-bdcc-4e07-b17f-a6b93734715a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.425034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd2e98-57a6-4170-a1ed-0def4fb97381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.425999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd2e98-57a6-4170-a1ed-0def4fb97381 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.426332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd2e98-57a6-4170-a1ed-0def4fb97381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.426614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdd2e98-57a6-4170-a1ed-0def4fb97381 admin admin] PATH_INFO: `/v3/projects/e0e8941e72d04beb86b3db8e89199f03/users/30f9e3a720a2498fb2691eaa46cb498d/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.444353 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd2e98-57a6-4170-a1ed-0def4fb97381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e8941e72d04beb86b3db8e89199f03, user_id=30f9e3a720a2498fb2691eaa46cb498d, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.447231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdd2e98-57a6-4170-a1ed-0def4fb97381 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.465247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1243/2498] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:26 2022] PUT /identity/v3/projects/e0e8941e72d04beb86b3db8e89199f03/users/30f9e3a720a2498fb2691eaa46cb498d/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:26.477375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc74e7-b1e3-4b14-9dd3-0281fda4ea46 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.477663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc74e7-b1e3-4b14-9dd3-0281fda4ea46 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.477950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fc74e7-b1e3-4b14-9dd3-0281fda4ea46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.508699 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d1fc74e7-b1e3-4b14-9dd3-0281fda4ea46 None None] MFA Rules not processed for user `30f9e3a720a2498fb2691eaa46cb498d`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:26.510256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d1fc74e7-b1e3-4b14-9dd3-0281fda4ea46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:26.603161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1256/2499] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:26.605044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd2e98-57a6-4170-a1ed-0def4fb97381 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.693555 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdd2e98-57a6-4170-a1ed-0def4fb97381 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.699023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeced91-874a-40a7-8e31-381c6c65b38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.699888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeced91-874a-40a7-8e31-381c6c65b38a admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.700240 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeced91-874a-40a7-8e31-381c6c65b38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.700584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeced91-874a-40a7-8e31-381c6c65b38a admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.701387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeced91-874a-40a7-8e31-381c6c65b38a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.703587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeced91-874a-40a7-8e31-381c6c65b38a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.745853 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1244/2500] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:26 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:26.750112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc74e7-b1e3-4b14-9dd3-0281fda4ea46 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.761957 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fc74e7-b1e3-4b14-9dd3-0281fda4ea46 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.839951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d52ca-f352-4ee0-b702-4105aa4d44dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.841204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d52ca-f352-4ee0-b702-4105aa4d44dd admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.841804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d52ca-f352-4ee0-b702-4105aa4d44dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.842307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d52ca-f352-4ee0-b702-4105aa4d44dd admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.843273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d52ca-f352-4ee0-b702-4105aa4d44dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.845434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d52ca-f352-4ee0-b702-4105aa4d44dd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.878286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeced91-874a-40a7-8e31-381c6c65b38a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.885337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1257/2501] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:26 2022] POST /identity/v3/users => generated 298 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:26.891746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeced91-874a-40a7-8e31-381c6c65b38a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.900522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37a050-02ee-47de-8461-0d5ce012044c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.901391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37a050-02ee-47de-8461-0d5ce012044c admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.901830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37a050-02ee-47de-8461-0d5ce012044c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.904040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b37a050-02ee-47de-8461-0d5ce012044c admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.904952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37a050-02ee-47de-8461-0d5ce012044c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.906794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b37a050-02ee-47de-8461-0d5ce012044c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:26.930870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1245/2502] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:26.935210 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d52ca-f352-4ee0-b702-4105aa4d44dd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:26.940561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:26.954627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d52ca-f352-4ee0-b702-4105aa4d44dd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:26.960268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535bded-9952-44d2-b6c4-7cd76a61696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:26.961174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535bded-9952-44d2-b6c4-7cd76a61696c admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:26.961453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535bded-9952-44d2-b6c4-7cd76a61696c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:26.961726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535bded-9952-44d2-b6c4-7cd76a61696c admin admin] PATH_INFO: `/v3/projects/95ec30427c174833ab60409f6205d2ce/users/16d47dc834b346eabf11768d481d4613/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:26.984523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535bded-9952-44d2-b6c4-7cd76a61696c admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ec30427c174833ab60409f6205d2ce, user_id=16d47dc834b346eabf11768d481d4613, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:26.987465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535bded-9952-44d2-b6c4-7cd76a61696c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:27.006311 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1258/2503] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:26 2022] PUT /identity/v3/projects/95ec30427c174833ab60409f6205d2ce/users/16d47dc834b346eabf11768d481d4613/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:27.010668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37a050-02ee-47de-8461-0d5ce012044c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:27.022618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b37a050-02ee-47de-8461-0d5ce012044c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:27.035197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:27.096424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5c1bd-660d-4f5a-a701-ce4a70631a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:27.097874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5c1bd-660d-4f5a-a701-ce4a70631a2d admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:27.098347 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5c1bd-660d-4f5a-a701-ce4a70631a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:27.099009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5c1bd-660d-4f5a-a701-ce4a70631a2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:27.100001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5c1bd-660d-4f5a-a701-ce4a70631a2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:27.101772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5c1bd-660d-4f5a-a701-ce4a70631a2d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:27.113734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1246/2504] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:27.118379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535bded-9952-44d2-b6c4-7cd76a61696c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:27.133913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535bded-9952-44d2-b6c4-7cd76a61696c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:27.144099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:27.145010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:27.145362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:27.145576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] PATH_INFO: `/v3/projects/95ec30427c174833ab60409f6205d2ce/users/16d47dc834b346eabf11768d481d4613/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:27.147848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ec30427c174833ab60409f6205d2ce, user_id=16d47dc834b346eabf11768d481d4613, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:27.151602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:27.163777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1259/2505] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:27 2022] PUT /identity/v3/projects/95ec30427c174833ab60409f6205d2ce/users/16d47dc834b346eabf11768d481d4613/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:27.174726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cab1d-ae52-4d17-ad63-418044d21b96 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:27.175059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cab1d-ae52-4d17-ad63-418044d21b96 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:27.175368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cab1d-ae52-4d17-ad63-418044d21b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:27.225342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-100cab1d-ae52-4d17-ad63-418044d21b96 None None] MFA Rules not processed for user `16d47dc834b346eabf11768d481d4613`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:27.226610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-100cab1d-ae52-4d17-ad63-418044d21b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:27.308689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1247/2506] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:27.320333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:27.322671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:27.392275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f00fb-f12d-4b9a-bc5f-abdb540aa24a admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:27.398931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:27.399639 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:27.399926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:27.400204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:27.474279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:27.475802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:27.485610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1260/2507] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:33.885158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cab1d-ae52-4d17-ad63-418044d21b96 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:33.896195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cab1d-ae52-4d17-ad63-418044d21b96 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:33.900405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c1585-0a6f-4f17-a8b8-598d125b8f4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:33.901105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c1585-0a6f-4f17-a8b8-598d125b8f4f service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:33.901388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c1585-0a6f-4f17-a8b8-598d125b8f4f service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:33.901658 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c1585-0a6f-4f17-a8b8-598d125b8f4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:33.911517 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c1585-0a6f-4f17-a8b8-598d125b8f4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:33.912985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06c1585-0a6f-4f17-a8b8-598d125b8f4f service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:33.922825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1248/2508] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:35.744988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:35.746587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:35.820567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b223f51-79e9-4dc0-976c-b4faa4ce765b service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:35.826386 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444b59ad-34d5-4fed-ac98-8c419a701967 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:35.827056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b59ad-34d5-4fed-ac98-8c419a701967 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:35.827395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b59ad-34d5-4fed-ac98-8c419a701967 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:35.827993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444b59ad-34d5-4fed-ac98-8c419a701967 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:35.830370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-444b59ad-34d5-4fed-ac98-8c419a701967 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:35.870810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:35.902946 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b59ad-34d5-4fed-ac98-8c419a701967 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:35.904775 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444b59ad-34d5-4fed-ac98-8c419a701967 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:35.914882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1261/2509] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:38.874927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c1585-0a6f-4f17-a8b8-598d125b8f4f service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:38.893243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c1585-0a6f-4f17-a8b8-598d125b8f4f service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:38.900287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:38.901625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:38.902349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:38.902945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] PATH_INFO: `/v3/users/30f9e3a720a2498fb2691eaa46cb498d` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:38.905060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] RBAC: Authorizing `identity:delete_user(user_id=30f9e3a720a2498fb2691eaa46cb498d)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:38.907918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:38.975234 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30f9e3a720a2498fb2691eaa46cb498d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:38.976356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30f9e3a720a2498fb2691eaa46cb498d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:38.978013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30f9e3a720a2498fb2691eaa46cb498d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:38.992897 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30f9e3a720a2498fb2691eaa46cb498d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.001822 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30f9e3a720a2498fb2691eaa46cb498d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.032531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:39.039410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1249/2510] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:38 2022] DELETE /identity/v3/users/30f9e3a720a2498fb2691eaa46cb498d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:39.044735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed9968-c5c7-4ea1-a6f9-78d76bce1b26 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.045199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed9968-c5c7-4ea1-a6f9-78d76bce1b26 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.045282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed9968-c5c7-4ea1-a6f9-78d76bce1b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.069841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-13ed9968-c5c7-4ea1-a6f9-78d76bce1b26 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:39.072646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-13ed9968-c5c7-4ea1-a6f9-78d76bce1b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:39.148871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1262/2511] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:39.158794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:39.160370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:39.232850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed9968-c5c7-4ea1-a6f9-78d76bce1b26 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:39.234419 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-13ed9968-c5c7-4ea1-a6f9-78d76bce1b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:39.255434 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed9968-c5c7-4ea1-a6f9-78d76bce1b26 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:39.257784 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68cff65-9cb4-4837-9920-c66e246379be admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:39.264514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:39.265539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7e8a-c708-4894-9ac0-0b7da318ec36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:39.265806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.266599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.267052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] PATH_INFO: `/v3/users/bfaaa367491745708334c5f35331bf76` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.267435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7e8a-c708-4894-9ac0-0b7da318ec36 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.267927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7e8a-c708-4894-9ac0-0b7da318ec36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.268258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8a7e8a-c708-4894-9ac0-0b7da318ec36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.268376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfaaa367491745708334c5f35331bf76)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:39.270293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:39.284917 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a7e8a-c708-4894-9ac0-0b7da318ec36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:39.286935 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8a7e8a-c708-4894-9ac0-0b7da318ec36 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:39.318069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1250/2512] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:39.343691 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfaaa367491745708334c5f35331bf76'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.355038 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfaaa367491745708334c5f35331bf76'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.359472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfaaa367491745708334c5f35331bf76'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.371539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfaaa367491745708334c5f35331bf76'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.372499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfaaa367491745708334c5f35331bf76'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.395459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1263/2513] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:39 2022] DELETE /identity/v3/users/bfaaa367491745708334c5f35331bf76 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:39.616137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7e8a-c708-4894-9ac0-0b7da318ec36 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:39.617397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:39.619516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:39.629310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8a7e8a-c708-4894-9ac0-0b7da318ec36 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:39.682871 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:39.694322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:39.695487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.695773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.696054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] PATH_INFO: `/v3/projects/9dd35ce1fcbd4038a0c556a45059c7d7` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.697368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dd35ce1fcbd4038a0c556a45059c7d7)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:39.699343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:39.703152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbfa87e-e0f9-4d2f-af62-689286dc253d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:39.707753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:39.708648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.708956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.709231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] PATH_INFO: `/v3/projects/e0e8941e72d04beb86b3db8e89199f03` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.710433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e8941e72d04beb86b3db8e89199f03)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:39.712881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:39.746298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd35ce1fcbd4038a0c556a45059c7d7', 'request_id': 'req-2e8583de-77f4-4231-b2a4-952252889ea0'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.755207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd35ce1fcbd4038a0c556a45059c7d7', 'request_id': 'req-2e8583de-77f4-4231-b2a4-952252889ea0'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.760923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e8941e72d04beb86b3db8e89199f03', 'request_id': 'req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.761278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd35ce1fcbd4038a0c556a45059c7d7', 'request_id': 'req-2e8583de-77f4-4231-b2a4-952252889ea0'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.772746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] The token cache is being invalidate because project 9dd35ce1fcbd4038a0c556a45059c7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:39.773102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.773663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-2e8583de-77f4-4231-b2a4-952252889ea0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.774728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e8941e72d04beb86b3db8e89199f03', 'request_id': 'req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.775483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1251/2514] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:39 2022] DELETE /identity/v3/projects/9dd35ce1fcbd4038a0c556a45059c7d7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:39.780162 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cf82c-46f9-430c-9cbc-f8c945c623f0 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.780447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cf82c-46f9-430c-9cbc-f8c945c623f0 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.780852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930cf82c-46f9-430c-9cbc-f8c945c623f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.782333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e8941e72d04beb86b3db8e89199f03', 'request_id': 'req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.796301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] The token cache is being invalidate because project e0e8941e72d04beb86b3db8e89199f03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:39.796590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.797203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.798885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1264/2515] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:39 2022] DELETE /identity/v3/projects/e0e8941e72d04beb86b3db8e89199f03 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:39.802075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-930cf82c-46f9-430c-9cbc-f8c945c623f0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:39.802075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-930cf82c-46f9-430c-9cbc-f8c945c623f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:39.802778 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:39.804050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:39.824955 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:39.859919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1252/2516] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:39.867902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89d481-db64-4c6f-8a7a-8b2a94009a75 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.868197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89d481-db64-4c6f-8a7a-8b2a94009a75 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.868471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89d481-db64-4c6f-8a7a-8b2a94009a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.872774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d28bc-3bf6-42fc-8185-d46d6a9bcafd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:39.877976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:39.878976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.880294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.880294 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] PATH_INFO: `/v3/users/16d47dc834b346eabf11768d481d4613` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.881118 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16d47dc834b346eabf11768d481d4613)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:39.883177 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:39.889649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3d89d481-db64-4c6f-8a7a-8b2a94009a75 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:39.891446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3d89d481-db64-4c6f-8a7a-8b2a94009a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:39.906552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1253/2517] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:39.909847 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89d481-db64-4c6f-8a7a-8b2a94009a75 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:39.919010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89d481-db64-4c6f-8a7a-8b2a94009a75 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:39.924011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:39.924678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.925578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.925578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.929380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:39.929380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:39.929380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:39.945076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1254/2518] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:20:39.946251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:39.956721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343841ae-c635-4577-8f38-6700a5feb5b0 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:39.961601 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:39.962572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:39.962865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:39.963141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:39.963873 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:39.965790 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:39.969008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16d47dc834b346eabf11768d481d4613'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.990276 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16d47dc834b346eabf11768d481d4613'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:39.991175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16d47dc834b346eabf11768d481d4613'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:40.002089 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16d47dc834b346eabf11768d481d4613'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:40.003500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16d47dc834b346eabf11768d481d4613'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:40.009393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1255/2519] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:39 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:40.011975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.016556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:40.016778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:40.031441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1265/2520] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:39 2022] DELETE /identity/v3/users/16d47dc834b346eabf11768d481d4613 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:40.105959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447e77b-d4bf-461d-b500-6063e312bf57 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.189040 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cf82f-4fb8-4592-95e4-785c078b0501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.189855 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cf82f-4fb8-4592-95e4-785c078b0501 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.190195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cf82f-4fb8-4592-95e4-785c078b0501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.190530 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684cf82f-4fb8-4592-95e4-785c078b0501 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.191311 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cf82f-4fb8-4592-95e4-785c078b0501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.193317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684cf82f-4fb8-4592-95e4-785c078b0501 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.227466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1256/2521] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/users => generated 285 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:40.228834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.246101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1bf5d-dcd2-4933-b86a-ec6d81c85f93 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.253938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9830d-1d97-4936-890e-8960e730b83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.255469 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9830d-1d97-4936-890e-8960e730b83d admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.255787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9830d-1d97-4936-890e-8960e730b83d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.256245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9830d-1d97-4936-890e-8960e730b83d admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.257107 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9830d-1d97-4936-890e-8960e730b83d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.258879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9830d-1d97-4936-890e-8960e730b83d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.272282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1266/2522] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:40.276348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cf82f-4fb8-4592-95e4-785c078b0501 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.293182 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684cf82f-4fb8-4592-95e4-785c078b0501 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.298758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.299848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.300134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.300401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.301548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.303380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.322921 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1257/2523] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:40.323287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9830d-1d97-4936-890e-8960e730b83d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.339213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9830d-1d97-4936-890e-8960e730b83d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.344958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7087fffa-436f-43c9-aa75-a19105baddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.346324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087fffa-436f-43c9-aa75-a19105baddfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.346851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087fffa-436f-43c9-aa75-a19105baddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.347468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7087fffa-436f-43c9-aa75-a19105baddfc admin admin] PATH_INFO: `/v3/projects/d207df2951d6460a9c0e3432e806e868/users/74426c0962b545ebae48a89ca8ddd157/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.369274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087fffa-436f-43c9-aa75-a19105baddfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d207df2951d6460a9c0e3432e806e868, user_id=74426c0962b545ebae48a89ca8ddd157, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.372362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7087fffa-436f-43c9-aa75-a19105baddfc admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.384512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.387390 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:40.392008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1267/2524] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:40 2022] PUT /identity/v3/projects/d207df2951d6460a9c0e3432e806e868/users/74426c0962b545ebae48a89ca8ddd157/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:40.403281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deb57b-1669-4605-8f48-e5bc7eebeb8a None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.403981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deb57b-1669-4605-8f48-e5bc7eebeb8a None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.404365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33deb57b-1669-4605-8f48-e5bc7eebeb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.413737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee6295-50c2-447f-a927-fa4b68e8c096 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.426376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:40.440113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-33deb57b-1669-4605-8f48-e5bc7eebeb8a None None] MFA Rules not processed for user `74426c0962b545ebae48a89ca8ddd157`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:40.442801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-33deb57b-1669-4605-8f48-e5bc7eebeb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:40.481973 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.482887 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.483199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.483508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] PATH_INFO: `/v3/projects/95ec30427c174833ab60409f6205d2ce` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.484724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ec30427c174833ab60409f6205d2ce)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.486642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.523226 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1268/2525] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:40.527770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33deb57b-1669-4605-8f48-e5bc7eebeb8a None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.538212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33deb57b-1669-4605-8f48-e5bc7eebeb8a None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.546410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ec30427c174833ab60409f6205d2ce', 'request_id': 'req-24c0560a-a001-4018-a832-4cb8913f55a0'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:40.555548 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ec30427c174833ab60409f6205d2ce', 'request_id': 'req-24c0560a-a001-4018-a832-4cb8913f55a0'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:40.561489 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ec30427c174833ab60409f6205d2ce', 'request_id': 'req-24c0560a-a001-4018-a832-4cb8913f55a0'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:40.573092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] The token cache is being invalidate because project 95ec30427c174833ab60409f6205d2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:40.573366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:40.574015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-24c0560a-a001-4018-a832-4cb8913f55a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:40.575649 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1258/2526] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:40 2022] DELETE /identity/v3/projects/95ec30427c174833ab60409f6205d2ce => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:40.581646 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20be2c-8ba9-43c9-b14f-16e648bfae3e None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.581776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20be2c-8ba9-43c9-b14f-16e648bfae3e None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.582045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20be2c-8ba9-43c9-b14f-16e648bfae3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.593536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.594618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.594896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.595167 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.595883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.597805 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.599467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-dd20be2c-8ba9-43c9-b14f-16e648bfae3e None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:40.600671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-dd20be2c-8ba9-43c9-b14f-16e648bfae3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:40.608647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1259/2527] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:40.617000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b615-1cdf-4a3e-b5e0-4b5ca43e790b None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.617287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b615-1cdf-4a3e-b5e0-4b5ca43e790b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.617572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1b615-1cdf-4a3e-b5e0-4b5ca43e790b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.627156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1269/2528] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:40.628263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.629920 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:40.635665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-97c1b615-1cdf-4a3e-b5e0-4b5ca43e790b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:40.636836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-97c1b615-1cdf-4a3e-b5e0-4b5ca43e790b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:40.663449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:DELETE=1,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:40.694397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf2e00-609f-4087-ad93-878769e0683e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.699094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec08b8e-445f-4b0d-9a3e-bf0055f86a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.699793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec08b8e-445f-4b0d-9a3e-bf0055f86a2b admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.700249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1260/2529] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:40.700365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec08b8e-445f-4b0d-9a3e-bf0055f86a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.700365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec08b8e-445f-4b0d-9a3e-bf0055f86a2b admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.701095 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec08b8e-445f-4b0d-9a3e-bf0055f86a2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.703019 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec08b8e-445f-4b0d-9a3e-bf0055f86a2b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.704372 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1b615-1cdf-4a3e-b5e0-4b5ca43e790b None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.714441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1b615-1cdf-4a3e-b5e0-4b5ca43e790b None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.719743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.720393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.720673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.720938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.721660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:40.721965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.723591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.725332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1270/2530] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:40.729191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec08b8e-445f-4b0d-9a3e-bf0055f86a2b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.735553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1261/2531] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:20:40.738754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec08b8e-445f-4b0d-9a3e-bf0055f86a2b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.740010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.743406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090a842-0a7a-4d25-a21c-cf7d1b1886dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.744046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090a842-0a7a-4d25-a21c-cf7d1b1886dd admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.744326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090a842-0a7a-4d25-a21c-cf7d1b1886dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.744726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090a842-0a7a-4d25-a21c-cf7d1b1886dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.745331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090a842-0a7a-4d25-a21c-cf7d1b1886dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.746926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7090a842-0a7a-4d25-a21c-cf7d1b1886dd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.749499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565422a8-8a58-4dc1-ba1f-de22550984b8 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.757744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a52ba-e776-4f83-b206-4b4496d07ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.758392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a52ba-e776-4f83-b206-4b4496d07ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.758672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a52ba-e776-4f83-b206-4b4496d07ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.758946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a52ba-e776-4f83-b206-4b4496d07ef3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.759896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a52ba-e776-4f83-b206-4b4496d07ef3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.760027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1271/2532] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:40.762443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013a52ba-e776-4f83-b206-4b4496d07ef3 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.764373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090a842-0a7a-4d25-a21c-cf7d1b1886dd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.774659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7090a842-0a7a-4d25-a21c-cf7d1b1886dd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.779892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a6042-5d25-4d92-a7ff-e357e5be0791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.780796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a6042-5d25-4d92-a7ff-e357e5be0791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.781881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a6042-5d25-4d92-a7ff-e357e5be0791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.781881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a6042-5d25-4d92-a7ff-e357e5be0791 admin admin] PATH_INFO: `/v3/projects/6d7339a640114f45adf47a55f4086687/users/37499c95aace4617b2b146b92fd300c4/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.800562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a6042-5d25-4d92-a7ff-e357e5be0791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7339a640114f45adf47a55f4086687, user_id=37499c95aace4617b2b146b92fd300c4, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.802197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1262/2533] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/projects => generated 359 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:40.803404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a6042-5d25-4d92-a7ff-e357e5be0791 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.806339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a52ba-e776-4f83-b206-4b4496d07ef3 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.810967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:40.818900 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1272/2534] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:40 2022] PUT /identity/v3/projects/6d7339a640114f45adf47a55f4086687/users/37499c95aace4617b2b146b92fd300c4/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:40.822874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a6042-5d25-4d92-a7ff-e357e5be0791 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.869111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013a52ba-e776-4f83-b206-4b4496d07ef3 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.881915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a6042-5d25-4d92-a7ff-e357e5be0791 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.886413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad16e0-e92f-48fa-ba6d-92a4df638fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.887076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad16e0-e92f-48fa-ba6d-92a4df638fbe admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.887388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad16e0-e92f-48fa-ba6d-92a4df638fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.887666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad16e0-e92f-48fa-ba6d-92a4df638fbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.888425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad16e0-e92f-48fa-ba6d-92a4df638fbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.890246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad16e0-e92f-48fa-ba6d-92a4df638fbe admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.900553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1273/2535] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:40.904765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad16e0-e92f-48fa-ba6d-92a4df638fbe admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.914681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad16e0-e92f-48fa-ba6d-92a4df638fbe admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.920024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed71d6e-3688-43c5-93cf-053eb76f37a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.920965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed71d6e-3688-43c5-93cf-053eb76f37a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.921252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed71d6e-3688-43c5-93cf-053eb76f37a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.921528 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed71d6e-3688-43c5-93cf-053eb76f37a9 admin admin] PATH_INFO: `/v3/projects/6d7339a640114f45adf47a55f4086687/users/37499c95aace4617b2b146b92fd300c4/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.923231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c405a-20bd-47a1-948b-70f4ed181b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:40.923741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed71d6e-3688-43c5-93cf-053eb76f37a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d7339a640114f45adf47a55f4086687, user_id=37499c95aace4617b2b146b92fd300c4, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.923933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c405a-20bd-47a1-948b-70f4ed181b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.924238 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c405a-20bd-47a1-948b-70f4ed181b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.924510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c405a-20bd-47a1-948b-70f4ed181b2e admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.925211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c405a-20bd-47a1-948b-70f4ed181b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:40.926389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed71d6e-3688-43c5-93cf-053eb76f37a9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.927114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c405a-20bd-47a1-948b-70f4ed181b2e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:40.937698 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1274/2536] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:40 2022] PUT /identity/v3/projects/6d7339a640114f45adf47a55f4086687/users/37499c95aace4617b2b146b92fd300c4/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:40.948255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a8f7-433c-4a12-a4a9-639d97364586 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:40.948548 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a8f7-433c-4a12-a4a9-639d97364586 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:40.948824 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64a8f7-433c-4a12-a4a9-639d97364586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:40.949169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1263/2537] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/users => generated 280 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:40.953054 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c405a-20bd-47a1-948b-70f4ed181b2e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:40.962763 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c405a-20bd-47a1-948b-70f4ed181b2e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:40.976301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6c64a8f7-433c-4a12-a4a9-639d97364586 None None] MFA Rules not processed for user `37499c95aace4617b2b146b92fd300c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:40.977519 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6c64a8f7-433c-4a12-a4a9-639d97364586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:41.024151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd513e6-344a-422d-b0b1-21cf281e858a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:41.024987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd513e6-344a-422d-b0b1-21cf281e858a admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.025266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd513e6-344a-422d-b0b1-21cf281e858a admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.025527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd513e6-344a-422d-b0b1-21cf281e858a admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.026260 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd513e6-344a-422d-b0b1-21cf281e858a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:41.027959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd513e6-344a-422d-b0b1-21cf281e858a admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:41.037342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:41.038775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1264/2538] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:41.042088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd513e6-344a-422d-b0b1-21cf281e858a admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:41.049022 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1275/2539] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:40 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:41.052356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd513e6-344a-422d-b0b1-21cf281e858a admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:41.056919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5df600-d2c3-40a1-943c-99ea4218c56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:41.057623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5df600-d2c3-40a1-943c-99ea4218c56e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.057903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5df600-d2c3-40a1-943c-99ea4218c56e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.058175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5df600-d2c3-40a1-943c-99ea4218c56e admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.058898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5df600-d2c3-40a1-943c-99ea4218c56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:41.059808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64a8f7-433c-4a12-a4a9-639d97364586 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:41.060600 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5df600-d2c3-40a1-943c-99ea4218c56e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:41.061366 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6c64a8f7-433c-4a12-a4a9-639d97364586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:41.070992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1265/2540] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:41.074995 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5df600-d2c3-40a1-943c-99ea4218c56e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:41.084482 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5df600-d2c3-40a1-943c-99ea4218c56e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:41.089150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ab87a-f263-4a57-94b4-866817891a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:41.090008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ab87a-f263-4a57-94b4-866817891a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.090285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ab87a-f263-4a57-94b4-866817891a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.090556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ab87a-f263-4a57-94b4-866817891a36 admin admin] PATH_INFO: `/v3/projects/5c349076c10646b7bc05e11b52c31b2c/users/fa402121ff4842d7a3f33ee0e0433690/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.105177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ab87a-f263-4a57-94b4-866817891a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c349076c10646b7bc05e11b52c31b2c, user_id=fa402121ff4842d7a3f33ee0e0433690, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:41.107984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ab87a-f263-4a57-94b4-866817891a36 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:41.118657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:41.123563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64a8f7-433c-4a12-a4a9-639d97364586 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:41.126237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1266/2541] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:41 2022] PUT /identity/v3/projects/5c349076c10646b7bc05e11b52c31b2c/users/fa402121ff4842d7a3f33ee0e0433690/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:41.138457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a2f7d-4016-4aef-b7da-f31bacbb8aef None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.138680 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a2f7d-4016-4aef-b7da-f31bacbb8aef None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.138956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7a2f7d-4016-4aef-b7da-f31bacbb8aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.171455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3d7a2f7d-4016-4aef-b7da-f31bacbb8aef None None] MFA Rules not processed for user `fa402121ff4842d7a3f33ee0e0433690`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:41.172683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3d7a2f7d-4016-4aef-b7da-f31bacbb8aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:41.181725 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e23bec-a515-47aa-b2b9-f7591f6cd08d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:41.182379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23bec-a515-47aa-b2b9-f7591f6cd08d service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.182660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23bec-a515-47aa-b2b9-f7591f6cd08d service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.182954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e23bec-a515-47aa-b2b9-f7591f6cd08d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.234950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1267/2542] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:41 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:41.241258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e23bec-a515-47aa-b2b9-f7591f6cd08d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:41.242759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e23bec-a515-47aa-b2b9-f7591f6cd08d service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:41.252599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1276/2543] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:41 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:20:41.266259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a2f7d-4016-4aef-b7da-f31bacbb8aef None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:41.267839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3d7a2f7d-4016-4aef-b7da-f31bacbb8aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:41.280381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7a2f7d-4016-4aef-b7da-f31bacbb8aef None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:41.284398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9152d791-c88d-49a0-856f-fb17f00ec2d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:41.285119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d791-c88d-49a0-856f-fb17f00ec2d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.285393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d791-c88d-49a0-856f-fb17f00ec2d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.285653 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9152d791-c88d-49a0-856f-fb17f00ec2d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.294992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152d791-c88d-49a0-856f-fb17f00ec2d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:41.296476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9152d791-c88d-49a0-856f-fb17f00ec2d3 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:41.305731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1268/2544] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:41 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:41.319838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e23bec-a515-47aa-b2b9-f7591f6cd08d service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:41.383803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e23bec-a515-47aa-b2b9-f7591f6cd08d service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:41.388541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:41.389378 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.389659 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.389974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] PATH_INFO: `/v3/users/fa402121ff4842d7a3f33ee0e0433690` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.391541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa402121ff4842d7a3f33ee0e0433690)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:41.393468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:41.395005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:41.442629 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa402121ff4842d7a3f33ee0e0433690'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:41.450628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa402121ff4842d7a3f33ee0e0433690'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:41.451569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa402121ff4842d7a3f33ee0e0433690'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:41.460702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa402121ff4842d7a3f33ee0e0433690'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:41.461546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa402121ff4842d7a3f33ee0e0433690'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:41.479401 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1277/2545] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:41 2022] DELETE /identity/v3/users/fa402121ff4842d7a3f33ee0e0433690 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:41.484299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e81c2f-1b85-4664-b25a-88978ce71c55 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.484704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e81c2f-1b85-4664-b25a-88978ce71c55 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.485073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e81c2f-1b85-4664-b25a-88978ce71c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.507448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-72e81c2f-1b85-4664-b25a-88978ce71c55 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:41.508900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-72e81c2f-1b85-4664-b25a-88978ce71c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:41.579468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1269/2546] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:41.589817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:41.591497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:41.657327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77653f1b-c475-4bd2-8ad3-b8fee22483e3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:41.661563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a08065-256d-47e7-9bf6-ea0c0b3ed606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:41.662283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a08065-256d-47e7-9bf6-ea0c0b3ed606 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.662628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a08065-256d-47e7-9bf6-ea0c0b3ed606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.662967 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a08065-256d-47e7-9bf6-ea0c0b3ed606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.675397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a08065-256d-47e7-9bf6-ea0c0b3ed606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:41.677084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a08065-256d-47e7-9bf6-ea0c0b3ed606 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:41.688820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1278/2547] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:41.965668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e81c2f-1b85-4664-b25a-88978ce71c55 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:41.967298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-72e81c2f-1b85-4664-b25a-88978ce71c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:41.984478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e81c2f-1b85-4664-b25a-88978ce71c55 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:41.990464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:41.991412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:41.991690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:41.991959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] PATH_INFO: `/v3/projects/5c349076c10646b7bc05e11b52c31b2c` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:41.993299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c349076c10646b7bc05e11b52c31b2c)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:41.996800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.054743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c349076c10646b7bc05e11b52c31b2c', 'request_id': 'req-00070abd-2ba6-4099-bc55-3ba8bf087861'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.064650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c349076c10646b7bc05e11b52c31b2c', 'request_id': 'req-00070abd-2ba6-4099-bc55-3ba8bf087861'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.070705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c349076c10646b7bc05e11b52c31b2c', 'request_id': 'req-00070abd-2ba6-4099-bc55-3ba8bf087861'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.078550 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:42.082950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] The token cache is being invalidate because project 5c349076c10646b7bc05e11b52c31b2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:42.083490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.084519 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-00070abd-2ba6-4099-bc55-3ba8bf087861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.087337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1270/2548] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:41 2022] DELETE /identity/v3/projects/5c349076c10646b7bc05e11b52c31b2c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:42.091203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56245-041b-46d2-bc69-c7e1e4e97cd6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.091591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56245-041b-46d2-bc69-c7e1e4e97cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.091813 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c56245-041b-46d2-bc69-c7e1e4e97cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.110255 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-35c56245-041b-46d2-bc69-c7e1e4e97cd6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:42.113347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-35c56245-041b-46d2-bc69-c7e1e4e97cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:42.185850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1279/2549] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:42.193591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8aa0-3b01-4971-a026-e5b325409dbc None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.193903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8aa0-3b01-4971-a026-e5b325409dbc None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.194179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cd8aa0-3b01-4971-a026-e5b325409dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.212403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-41cd8aa0-3b01-4971-a026-e5b325409dbc None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:42.213596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-41cd8aa0-3b01-4971-a026-e5b325409dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:42.221604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1271/2550] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:42.226219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c56245-041b-46d2-bc69-c7e1e4e97cd6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.236996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c56245-041b-46d2-bc69-c7e1e4e97cd6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.241716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.242382 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.242783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.243085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.243887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:42.244211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.249412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.260977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1280/2551] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:42.262424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd8aa0-3b01-4971-a026-e5b325409dbc None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.273033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cd8aa0-3b01-4971-a026-e5b325409dbc None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.278056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812b46e-4ddf-4dc4-ac17-a5f26dbfc514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.278792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812b46e-4ddf-4dc4-ac17-a5f26dbfc514 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.279098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812b46e-4ddf-4dc4-ac17-a5f26dbfc514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.279413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c812b46e-4ddf-4dc4-ac17-a5f26dbfc514 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.280126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812b46e-4ddf-4dc4-ac17-a5f26dbfc514 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.282028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c812b46e-4ddf-4dc4-ac17-a5f26dbfc514 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.315664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1272/2552] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:42 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:42.316985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.330688 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb4badd-6954-496d-83a4-ab70e5339d63 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.335147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:42.397560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85af6-7846-46b5-9c1e-e3d78284b1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.398244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85af6-7846-46b5-9c1e-e3d78284b1bc admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.398525 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85af6-7846-46b5-9c1e-e3d78284b1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.398801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c85af6-7846-46b5-9c1e-e3d78284b1bc admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.399523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c85af6-7846-46b5-9c1e-e3d78284b1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.401572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c85af6-7846-46b5-9c1e-e3d78284b1bc admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.423309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1281/2553] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:42 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:42.427766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812b46e-4ddf-4dc4-ac17-a5f26dbfc514 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.438803 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c812b46e-4ddf-4dc4-ac17-a5f26dbfc514 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.443458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab1f7a-7e82-4984-8af4-a5899d48453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.444126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1f7a-7e82-4984-8af4-a5899d48453e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.444403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1f7a-7e82-4984-8af4-a5899d48453e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.444673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ab1f7a-7e82-4984-8af4-a5899d48453e admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.445403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab1f7a-7e82-4984-8af4-a5899d48453e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.447005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ab1f7a-7e82-4984-8af4-a5899d48453e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.460589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1273/2554] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:42.464280 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85af6-7846-46b5-9c1e-e3d78284b1bc admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.475188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c85af6-7846-46b5-9c1e-e3d78284b1bc admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.479851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3697e78-e0ea-4c9c-b9f7-6463f3e4dc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.480535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3697e78-e0ea-4c9c-b9f7-6463f3e4dc68 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.480814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3697e78-e0ea-4c9c-b9f7-6463f3e4dc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.481086 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3697e78-e0ea-4c9c-b9f7-6463f3e4dc68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.481826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3697e78-e0ea-4c9c-b9f7-6463f3e4dc68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.483457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3697e78-e0ea-4c9c-b9f7-6463f3e4dc68 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.493661 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1282/2555] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:42.499154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab1f7a-7e82-4984-8af4-a5899d48453e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.511770 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ab1f7a-7e82-4984-8af4-a5899d48453e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.516935 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.517843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.518127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.518401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] PATH_INFO: `/v3/projects/5e6157844f21442aac26dcb71092e6aa/users/ce83a48dd5874defb52933070d8a9b7e/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.534392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6157844f21442aac26dcb71092e6aa, user_id=ce83a48dd5874defb52933070d8a9b7e, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.537106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.551915 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1274/2556] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:42 2022] PUT /identity/v3/projects/5e6157844f21442aac26dcb71092e6aa/users/ce83a48dd5874defb52933070d8a9b7e/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:42.562517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c975b31-cacf-46f3-a856-704a70c46816 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.562803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c975b31-cacf-46f3-a856-704a70c46816 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.563080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c975b31-cacf-46f3-a856-704a70c46816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.590625 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-7c975b31-cacf-46f3-a856-704a70c46816 None None] MFA Rules not processed for user `ce83a48dd5874defb52933070d8a9b7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:42.591884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7c975b31-cacf-46f3-a856-704a70c46816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:42.660605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1283/2557] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:42.665290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c975b31-cacf-46f3-a856-704a70c46816 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.665430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.667179 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:42.733997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c975b31-cacf-46f3-a856-704a70c46816 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.738697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4523e357-6d54-4209-8dde-a3834c832dd0 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.740695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29884944-c8ff-4c4e-8241-114acc777536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.741685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29884944-c8ff-4c4e-8241-114acc777536 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.742391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29884944-c8ff-4c4e-8241-114acc777536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.742897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29884944-c8ff-4c4e-8241-114acc777536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.743379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.743977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29884944-c8ff-4c4e-8241-114acc777536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.744262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.744545 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.744824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] PATH_INFO: `/v3/users/74426c0962b545ebae48a89ca8ddd157` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.746028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74426c0962b545ebae48a89ca8ddd157)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.746939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29884944-c8ff-4c4e-8241-114acc777536 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.748036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.792219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1284/2558] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:42 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:42.796314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29884944-c8ff-4c4e-8241-114acc777536 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.804266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:42.819272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74426c0962b545ebae48a89ca8ddd157'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.820246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74426c0962b545ebae48a89ca8ddd157'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.820863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74426c0962b545ebae48a89ca8ddd157'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.830812 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74426c0962b545ebae48a89ca8ddd157'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.839274 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b0663295-5e5e-4f37-9cf7-3d5b573e1f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74426c0962b545ebae48a89ca8ddd157'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:42.858097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1275/2559] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:42 2022] DELETE /identity/v3/users/74426c0962b545ebae48a89ca8ddd157 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:42.862459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9701a-e946-4d8a-8346-2aefb00f7b89 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.862843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9701a-e946-4d8a-8346-2aefb00f7b89 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.863216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9701a-e946-4d8a-8346-2aefb00f7b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.870909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29884944-c8ff-4c4e-8241-114acc777536 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.881895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-15e9701a-e946-4d8a-8346-2aefb00f7b89 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:42.883107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-15e9701a-e946-4d8a-8346-2aefb00f7b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:42.892426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:42.926979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.927834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.928222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.928662 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.929435 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.931418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.941718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1276/2560] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:42.951835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9701a-e946-4d8a-8346-2aefb00f7b89 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.953280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-15e9701a-e946-4d8a-8346-2aefb00f7b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:42.954206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1285/2561] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:42 2022] POST /identity/v3/users => generated 292 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:42.955065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.956602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:42.967703 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c16386-d34d-41f2-b896-f51e9768cec3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:42.972166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34252f57-2ff9-472b-8731-20554182004d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:42.972801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34252f57-2ff9-472b-8731-20554182004d admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:42.973079 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34252f57-2ff9-472b-8731-20554182004d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:42.973351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34252f57-2ff9-472b-8731-20554182004d admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:42.974074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34252f57-2ff9-472b-8731-20554182004d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:42.975690 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34252f57-2ff9-472b-8731-20554182004d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:42.985560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1286/2562] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:42.989392 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34252f57-2ff9-472b-8731-20554182004d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:42.999504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34252f57-2ff9-472b-8731-20554182004d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.004718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17683ba-612a-41f3-875d-16ee54414250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.005347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17683ba-612a-41f3-875d-16ee54414250 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.005624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17683ba-612a-41f3-875d-16ee54414250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.005898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17683ba-612a-41f3-875d-16ee54414250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.006613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17683ba-612a-41f3-875d-16ee54414250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.008301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17683ba-612a-41f3-875d-16ee54414250 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.011878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9701a-e946-4d8a-8346-2aefb00f7b89 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.015793 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f312d5-a5d7-4f7c-8042-8e8830b1f332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.016884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f312d5-a5d7-4f7c-8042-8e8830b1f332 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.017470 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f312d5-a5d7-4f7c-8042-8e8830b1f332 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.017863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f312d5-a5d7-4f7c-8042-8e8830b1f332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.020444 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1287/2563] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:43.024322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17683ba-612a-41f3-875d-16ee54414250 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.028567 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f312d5-a5d7-4f7c-8042-8e8830b1f332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.030298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f312d5-a5d7-4f7c-8042-8e8830b1f332 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.034216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17683ba-612a-41f3-875d-16ee54414250 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.039139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfd0c5-93d5-4af7-9c53-88a9d75d502e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.041309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfd0c5-93d5-4af7-9c53-88a9d75d502e admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.041309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfd0c5-93d5-4af7-9c53-88a9d75d502e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.041527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1277/2564] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:20:43.041622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dfd0c5-93d5-4af7-9c53-88a9d75d502e admin admin] PATH_INFO: `/v3/projects/b490a608f24447a089fece717482333d/users/5b65457e7b764bb19e94a18d404a9fcc/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.058160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfd0c5-93d5-4af7-9c53-88a9d75d502e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b490a608f24447a089fece717482333d, user_id=5b65457e7b764bb19e94a18d404a9fcc, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.062341 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dfd0c5-93d5-4af7-9c53-88a9d75d502e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.080971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1288/2565] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:43 2022] PUT /identity/v3/projects/b490a608f24447a089fece717482333d/users/5b65457e7b764bb19e94a18d404a9fcc/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:43.092137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a45a-da8d-449c-840d-9ce87e80f4a6 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.092427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a45a-da8d-449c-840d-9ce87e80f4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.092699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20a45a-da8d-449c-840d-9ce87e80f4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.119771 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-6a20a45a-da8d-449c-840d-9ce87e80f4a6 None None] MFA Rules not processed for user `5b65457e7b764bb19e94a18d404a9fcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:43.121082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-6a20a45a-da8d-449c-840d-9ce87e80f4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:43.195475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1278/2566] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:43.198934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfd0c5-93d5-4af7-9c53-88a9d75d502e admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.276450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:43.288764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dfd0c5-93d5-4af7-9c53-88a9d75d502e admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.297770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.298412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.298754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.299045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.299855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.301801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.343582 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1289/2567] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:43 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:43.348861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20a45a-da8d-449c-840d-9ce87e80f4a6 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.351062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.353169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:43.363431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20a45a-da8d-449c-840d-9ce87e80f4a6 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.428715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:43.443885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299f34-e406-4fc9-9f26-f4121185f3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.444780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299f34-e406-4fc9-9f26-f4121185f3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.445067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299f34-e406-4fc9-9f26-f4121185f3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.445342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23299f34-e406-4fc9-9f26-f4121185f3b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.446098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23299f34-e406-4fc9-9f26-f4121185f3b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.449291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23299f34-e406-4fc9-9f26-f4121185f3b8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.453147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99fd4d4-9b2f-4c14-8225-ac4f7d903446 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.461485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.462442 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.462887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.463220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] PATH_INFO: `/v3/projects/d207df2951d6460a9c0e3432e806e868` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.464651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d207df2951d6460a9c0e3432e806e868)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.466630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.477976 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1279/2568] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:43 2022] POST /identity/v3/users => generated 298 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:43.482028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299f34-e406-4fc9-9f26-f4121185f3b8 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.493427 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23299f34-e406-4fc9-9f26-f4121185f3b8 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.498262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75889-82c9-4c4d-9845-dd3bce7345a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.498998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75889-82c9-4c4d-9845-dd3bce7345a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.499406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75889-82c9-4c4d-9845-dd3bce7345a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.499896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a75889-82c9-4c4d-9845-dd3bce7345a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.500718 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a75889-82c9-4c4d-9845-dd3bce7345a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.502625 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a75889-82c9-4c4d-9845-dd3bce7345a5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.517740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1280/2569] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:43.521878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75889-82c9-4c4d-9845-dd3bce7345a5 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.532829 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd207df2951d6460a9c0e3432e806e868', 'request_id': 'req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.538867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a75889-82c9-4c4d-9845-dd3bce7345a5 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.540314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd207df2951d6460a9c0e3432e806e868', 'request_id': 'req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.547247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd207df2951d6460a9c0e3432e806e868', 'request_id': 'req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.568126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] The token cache is being invalidate because project d207df2951d6460a9c0e3432e806e868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:43.568477 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.571245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.573482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1290/2570] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:43 2022] DELETE /identity/v3/projects/d207df2951d6460a9c0e3432e806e868 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:43.579806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.581532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:43.615225 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.616155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.616438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.616714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] PATH_INFO: `/v3/projects/082371c0e5ce4871918fa6a9ab5ca689/users/3baa07439f174053a18714bcec00b90d/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.632464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=082371c0e5ce4871918fa6a9ab5ca689, user_id=3baa07439f174053a18714bcec00b90d, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.635710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.650657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1281/2571] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:43 2022] PUT /identity/v3/projects/082371c0e5ce4871918fa6a9ab5ca689/users/3baa07439f174053a18714bcec00b90d/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:43.654477 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.656215 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:43.656881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceb7d2a-ea3f-4b03-9316-5112bfba5ca0 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.670794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:43.722016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.722814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.723093 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.723443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] PATH_INFO: `/v3/users/37499c95aace4617b2b146b92fd300c4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.724666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=37499c95aace4617b2b146b92fd300c4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.724852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b395855-bb30-4b7c-88f9-c1afa48a32ce admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.726580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.730035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11259a07-8856-4ce1-a907-089a3eb3bba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.730690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259a07-8856-4ce1-a907-089a3eb3bba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.730958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259a07-8856-4ce1-a907-089a3eb3bba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.731237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11259a07-8856-4ce1-a907-089a3eb3bba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.732049 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11259a07-8856-4ce1-a907-089a3eb3bba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.733659 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11259a07-8856-4ce1-a907-089a3eb3bba9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.744287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1282/2572] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:43.748309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11259a07-8856-4ce1-a907-089a3eb3bba9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.758877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11259a07-8856-4ce1-a907-089a3eb3bba9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:43.764032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbe5a0-8c5f-43b3-a05e-231936f55e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:43.765046 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe5a0-8c5f-43b3-a05e-231936f55e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.765383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe5a0-8c5f-43b3-a05e-231936f55e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.765719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbe5a0-8c5f-43b3-a05e-231936f55e8d admin admin] PATH_INFO: `/v3/projects/082371c0e5ce4871918fa6a9ab5ca689/users/3baa07439f174053a18714bcec00b90d/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.768161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbe5a0-8c5f-43b3-a05e-231936f55e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=082371c0e5ce4871918fa6a9ab5ca689, user_id=3baa07439f174053a18714bcec00b90d, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:43.770842 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbe5a0-8c5f-43b3-a05e-231936f55e8d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:43.785661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1283/2573] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:43 2022] PUT /identity/v3/projects/082371c0e5ce4871918fa6a9ab5ca689/users/3baa07439f174053a18714bcec00b90d/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:43.795990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb151-4d63-4489-b85c-2e4c97d217d9 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:43.796098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb151-4d63-4489-b85c-2e4c97d217d9 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:43.796402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6bb151-4d63-4489-b85c-2e4c97d217d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:43.797436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37499c95aace4617b2b146b92fd300c4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.805713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37499c95aace4617b2b146b92fd300c4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.806550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37499c95aace4617b2b146b92fd300c4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.816371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37499c95aace4617b2b146b92fd300c4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.817592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37499c95aace4617b2b146b92fd300c4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:43.825753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0e6bb151-4d63-4489-b85c-2e4c97d217d9 None None] MFA Rules not processed for user `3baa07439f174053a18714bcec00b90d`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:43.826950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0e6bb151-4d63-4489-b85c-2e4c97d217d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:43.835643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1291/2574] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:43 2022] DELETE /identity/v3/users/37499c95aace4617b2b146b92fd300c4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:43.847912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:43.913869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1284/2575] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:43.928889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:43.930712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:44.021327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd000ce-2435-4b68-83e8-90b926f0ee2a admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:44.026228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e9d9d-267f-4baa-a83a-1c9367be0b99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:44.026982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e9d9d-267f-4baa-a83a-1c9367be0b99 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:44.027648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e9d9d-267f-4baa-a83a-1c9367be0b99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:44.027648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e9d9d-267f-4baa-a83a-1c9367be0b99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:44.030040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e55e9d9d-267f-4baa-a83a-1c9367be0b99 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:44.088776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:44.111772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e9d9d-267f-4baa-a83a-1c9367be0b99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:44.113402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e9d9d-267f-4baa-a83a-1c9367be0b99 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:44.124273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1292/2576] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:20:44.201736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bb151-4d63-4489-b85c-2e4c97d217d9 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:44.203299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0e6bb151-4d63-4489-b85c-2e4c97d217d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:44.306865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6bb151-4d63-4489-b85c-2e4c97d217d9 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:44.387671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:44.388584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:44.388894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:44.389163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] PATH_INFO: `/v3/projects/6d7339a640114f45adf47a55f4086687` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:44.392089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d7339a640114f45adf47a55f4086687)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:44.395375 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:44.510606 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d7339a640114f45adf47a55f4086687', 'request_id': 'req-238a17f4-b59f-4933-b8ea-16f74782f872'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:44.540278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d7339a640114f45adf47a55f4086687', 'request_id': 'req-238a17f4-b59f-4933-b8ea-16f74782f872'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:44.547374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d7339a640114f45adf47a55f4086687', 'request_id': 'req-238a17f4-b59f-4933-b8ea-16f74782f872'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:44.551104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:44.571626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] The token cache is being invalidate because project 6d7339a640114f45adf47a55f4086687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:44.571932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:44.572568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-238a17f4-b59f-4933-b8ea-16f74782f872 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:44.583608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1285/2577] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:44 2022] DELETE /identity/v3/projects/6d7339a640114f45adf47a55f4086687 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:44.589013 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a1201-9709-4c04-9df9-560bfa064b6d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:44.591615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a1201-9709-4c04-9df9-560bfa064b6d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:44.592123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6a1201-9709-4c04-9df9-560bfa064b6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:44.620532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-de6a1201-9709-4c04-9df9-560bfa064b6d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:44.624052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-de6a1201-9709-4c04-9df9-560bfa064b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:44.712077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1293/2578] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:44.719025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803a454-4b5c-4b61-aff3-3318107012c7 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:44.719334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803a454-4b5c-4b61-aff3-3318107012c7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:44.719618 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803a454-4b5c-4b61-aff3-3318107012c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:44.741448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b803a454-4b5c-4b61-aff3-3318107012c7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:44.742775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b803a454-4b5c-4b61-aff3-3318107012c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:44.752363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1286/2579] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:44.756878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a1201-9709-4c04-9df9-560bfa064b6d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:44.767568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6a1201-9709-4c04-9df9-560bfa064b6d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:44.776218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:44.777201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:44.777709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:44.778513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:44.779592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:44.779993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:44.781834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:44.805680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1294/2580] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:44.811320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803a454-4b5c-4b61-aff3-3318107012c7 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:44.823966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803a454-4b5c-4b61-aff3-3318107012c7 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:44.832901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f6b65-1385-4f75-bbf6-86b16780bb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:44.833578 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f6b65-1385-4f75-bbf6-86b16780bb07 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:44.833859 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f6b65-1385-4f75-bbf6-86b16780bb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:44.834133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14f6b65-1385-4f75-bbf6-86b16780bb07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:44.834834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f6b65-1385-4f75-bbf6-86b16780bb07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:44.836893 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14f6b65-1385-4f75-bbf6-86b16780bb07 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:44.868920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1287/2581] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:44 2022] POST /identity/v3/projects => generated 393 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:44.873663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:44.885556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687477a8-9ba4-4248-a23e-f9dfbb5b886c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:44.953681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae48d65-659d-4abe-b367-b4cbbeaae3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:44.954513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48d65-659d-4abe-b367-b4cbbeaae3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:44.954889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48d65-659d-4abe-b367-b4cbbeaae3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:44.955325 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae48d65-659d-4abe-b367-b4cbbeaae3a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:44.956078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae48d65-659d-4abe-b367-b4cbbeaae3a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:44.958075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae48d65-659d-4abe-b367-b4cbbeaae3a9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:44.982186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1295/2582] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:44 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:44.983681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f6b65-1385-4f75-bbf6-86b16780bb07 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:44.994811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14f6b65-1385-4f75-bbf6-86b16780bb07 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:44.999304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3183e3-5643-46b8-845a-6ba8fb203681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:44.999987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3183e3-5643-46b8-845a-6ba8fb203681 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.000357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3183e3-5643-46b8-845a-6ba8fb203681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.000636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3183e3-5643-46b8-845a-6ba8fb203681 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.001365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3183e3-5643-46b8-845a-6ba8fb203681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.002966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3183e3-5643-46b8-845a-6ba8fb203681 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.013615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1288/2583] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:45.018706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae48d65-659d-4abe-b367-b4cbbeaae3a9 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.032157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae48d65-659d-4abe-b367-b4cbbeaae3a9 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.040058 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa94059-b078-45c9-bbd5-5e7ad5f6b392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.040853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94059-b078-45c9-bbd5-5e7ad5f6b392 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.041204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94059-b078-45c9-bbd5-5e7ad5f6b392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.041547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa94059-b078-45c9-bbd5-5e7ad5f6b392 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.042360 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa94059-b078-45c9-bbd5-5e7ad5f6b392 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.044074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa94059-b078-45c9-bbd5-5e7ad5f6b392 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.058664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1296/2584] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:45.062776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3183e3-5643-46b8-845a-6ba8fb203681 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.073642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3183e3-5643-46b8-845a-6ba8fb203681 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.078753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1594471-e5fa-4d1c-9192-7e5aa308f0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.080063 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1594471-e5fa-4d1c-9192-7e5aa308f0bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.081041 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1594471-e5fa-4d1c-9192-7e5aa308f0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.081350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1594471-e5fa-4d1c-9192-7e5aa308f0bf admin admin] PATH_INFO: `/v3/projects/d6f560e71a42428ea4087c3e06350e88/users/24d8c2c9cd004e1e862e5a8ef85e9c9d/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.107159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1594471-e5fa-4d1c-9192-7e5aa308f0bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6f560e71a42428ea4087c3e06350e88, user_id=24d8c2c9cd004e1e862e5a8ef85e9c9d, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.110165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1594471-e5fa-4d1c-9192-7e5aa308f0bf admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.127899 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1289/2585] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:45 2022] PUT /identity/v3/projects/d6f560e71a42428ea4087c3e06350e88/users/24d8c2c9cd004e1e862e5a8ef85e9c9d/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:45.139454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bd2a-e2bb-4bd5-810a-280537ec8cef None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.140035 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bd2a-e2bb-4bd5-810a-280537ec8cef None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.140317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618bd2a-e2bb-4bd5-810a-280537ec8cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.171565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3618bd2a-e2bb-4bd5-810a-280537ec8cef None None] MFA Rules not processed for user `24d8c2c9cd004e1e862e5a8ef85e9c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:45.172799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3618bd2a-e2bb-4bd5-810a-280537ec8cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:45.232977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:45.261708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1297/2586] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:45.266791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1594471-e5fa-4d1c-9192-7e5aa308f0bf admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.344567 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1594471-e5fa-4d1c-9192-7e5aa308f0bf admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.349627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d23b8-93c7-4c38-89af-bd273bcc1548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.350412 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d23b8-93c7-4c38-89af-bd273bcc1548 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.350823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d23b8-93c7-4c38-89af-bd273bcc1548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.351193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d23b8-93c7-4c38-89af-bd273bcc1548 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.352019 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d23b8-93c7-4c38-89af-bd273bcc1548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.354209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d23b8-93c7-4c38-89af-bd273bcc1548 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.388094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1290/2587] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:45 2022] POST /identity/v3/projects => generated 391 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:45.392873 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618bd2a-e2bb-4bd5-810a-280537ec8cef None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.405195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618bd2a-e2bb-4bd5-810a-280537ec8cef None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.468359 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9b52b-73d9-4dba-86e2-8988bb9c8e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.469040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9b52b-73d9-4dba-86e2-8988bb9c8e39 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.469322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9b52b-73d9-4dba-86e2-8988bb9c8e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.469598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9b52b-73d9-4dba-86e2-8988bb9c8e39 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.470318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9b52b-73d9-4dba-86e2-8988bb9c8e39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.472317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9b52b-73d9-4dba-86e2-8988bb9c8e39 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.493887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1298/2588] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:45 2022] POST /identity/v3/users => generated 302 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:45.502584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d23b8-93c7-4c38-89af-bd273bcc1548 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.517374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d23b8-93c7-4c38-89af-bd273bcc1548 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.522521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0ba26-5f4b-4238-80e8-ca51007fab61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.523436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0ba26-5f4b-4238-80e8-ca51007fab61 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.523714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0ba26-5f4b-4238-80e8-ca51007fab61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.523993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec0ba26-5f4b-4238-80e8-ca51007fab61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.524737 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0ba26-5f4b-4238-80e8-ca51007fab61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.526512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec0ba26-5f4b-4238-80e8-ca51007fab61 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.541923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1291/2589] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:45.542804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9b52b-73d9-4dba-86e2-8988bb9c8e39 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.558057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9b52b-73d9-4dba-86e2-8988bb9c8e39 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.565658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976221-673e-44c6-b691-618be95edd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.566792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976221-673e-44c6-b691-618be95edd77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.567252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976221-673e-44c6-b691-618be95edd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.567739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89976221-673e-44c6-b691-618be95edd77 admin admin] PATH_INFO: `/v3/projects/336bf74a2c1b4b7d8a83a1c194dbb9cb/users/6b56a4151bec459da65e3fef1f66cbfb/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.595352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89976221-673e-44c6-b691-618be95edd77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=336bf74a2c1b4b7d8a83a1c194dbb9cb, user_id=6b56a4151bec459da65e3fef1f66cbfb, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.595352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89976221-673e-44c6-b691-618be95edd77 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.616236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1299/2590] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:45 2022] PUT /identity/v3/projects/336bf74a2c1b4b7d8a83a1c194dbb9cb/users/6b56a4151bec459da65e3fef1f66cbfb/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:45.620211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0ba26-5f4b-4238-80e8-ca51007fab61 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.637323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec0ba26-5f4b-4238-80e8-ca51007fab61 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.701937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd9d40-172c-469b-9cad-f2272763d678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.702789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd9d40-172c-469b-9cad-f2272763d678 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.703163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd9d40-172c-469b-9cad-f2272763d678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.703560 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd9d40-172c-469b-9cad-f2272763d678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.704408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd9d40-172c-469b-9cad-f2272763d678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.706238 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fd9d40-172c-469b-9cad-f2272763d678 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.718404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1292/2591] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:45.721663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976221-673e-44c6-b691-618be95edd77 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.733815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89976221-673e-44c6-b691-618be95edd77 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.743140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.744097 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.744372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.744645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] PATH_INFO: `/v3/projects/336bf74a2c1b4b7d8a83a1c194dbb9cb/users/6b56a4151bec459da65e3fef1f66cbfb/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.746644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] RBAC: Authorizing `identity:create_grant(project_id=336bf74a2c1b4b7d8a83a1c194dbb9cb, user_id=6b56a4151bec459da65e3fef1f66cbfb, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.749537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.762459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1300/2592] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:45 2022] PUT /identity/v3/projects/336bf74a2c1b4b7d8a83a1c194dbb9cb/users/6b56a4151bec459da65e3fef1f66cbfb/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:45.778176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80d95-f185-4e86-a38f-c7c6976cfd2a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.778831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80d95-f185-4e86-a38f-c7c6976cfd2a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.779430 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80d95-f185-4e86-a38f-c7c6976cfd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.786923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:45.792795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.795516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:45.818684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-79d80d95-f185-4e86-a38f-c7c6976cfd2a None None] MFA Rules not processed for user `6b56a4151bec459da65e3fef1f66cbfb`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:45.820069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-79d80d95-f185-4e86-a38f-c7c6976cfd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:45.873441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba12c8-a543-4199-b0ff-ec04be6fa9af admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.878755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.879873 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.880422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.880705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.883545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:45.901685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1293/2593] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:45.912684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d80d95-f185-4e86-a38f-c7c6976cfd2a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:45.914165 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-79d80d95-f185-4e86-a38f-c7c6976cfd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:45.926981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d80d95-f185-4e86-a38f-c7c6976cfd2a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:45.931038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60da8c-66dd-4e0e-934a-cd06f857eb38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:45.931828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60da8c-66dd-4e0e-934a-cd06f857eb38 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:45.932174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60da8c-66dd-4e0e-934a-cd06f857eb38 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:45.932514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60da8c-66dd-4e0e-934a-cd06f857eb38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:45.941014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:45.943278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60da8c-66dd-4e0e-934a-cd06f857eb38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.945304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a60da8c-66dd-4e0e-934a-cd06f857eb38 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.952859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:45.954620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:45.957051 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1294/2594] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:45.965931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1301/2595] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:46.012104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60da8c-66dd-4e0e-934a-cd06f857eb38 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:46.022513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60da8c-66dd-4e0e-934a-cd06f857eb38 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:46.026656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1e48b-4207-479d-a912-1f6dc54fa099 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:46.027337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e48b-4207-479d-a912-1f6dc54fa099 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:46.027616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e48b-4207-479d-a912-1f6dc54fa099 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:46.027886 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1e48b-4207-479d-a912-1f6dc54fa099 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:46.097576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1e48b-4207-479d-a912-1f6dc54fa099 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:46.099103 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e1e48b-4207-479d-a912-1f6dc54fa099 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:46.108283 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1295/2596] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:46.915587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:46.924867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da9a95a-bab0-4587-94ee-46402586c30c service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:46.929377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:46.930020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:46.930301 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:46.930574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:46.932814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:46.998745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:47.000205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:47.009818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1302/2597] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:46 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:47.863876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e86c-7d74-43c8-b672-5360ee0554a7 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:47.864398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e86c-7d74-43c8-b672-5360ee0554a7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:47.864398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b381e86c-7d74-43c8-b672-5360ee0554a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:47.883415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-b381e86c-7d74-43c8-b672-5360ee0554a7 None None] MFA Rules not processed for user `661271a657a548468ff85fe298e0c5f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:47.884589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b381e86c-7d74-43c8-b672-5360ee0554a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:47.940923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1296/2598] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:49.898583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:49.900245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:49.972810 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69698e99-4800-4dd6-80c2-0c3f1e33da7b service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:49.978575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84bfc2-8627-4994-908f-a9a02d97842e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:49.979345 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84bfc2-8627-4994-908f-a9a02d97842e service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:49.979631 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84bfc2-8627-4994-908f-a9a02d97842e service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:49.979905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84bfc2-8627-4994-908f-a9a02d97842e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:49.981793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9c84bfc2-8627-4994-908f-a9a02d97842e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:50.047945 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84bfc2-8627-4994-908f-a9a02d97842e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:50.049544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84bfc2-8627-4994-908f-a9a02d97842e service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:50.059646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1303/2599] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:50.598248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381e86c-7d74-43c8-b672-5360ee0554a7 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:50.609522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b381e86c-7d74-43c8-b672-5360ee0554a7 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:50.614417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:50.615405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:50.616034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:50.616034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] PATH_INFO: `/v3/users/24d8c2c9cd004e1e862e5a8ef85e9c9d` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:50.618881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24d8c2c9cd004e1e862e5a8ef85e9c9d)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:50.620871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:50.635446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:50.682591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24d8c2c9cd004e1e862e5a8ef85e9c9d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:50.683252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24d8c2c9cd004e1e862e5a8ef85e9c9d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:50.684106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24d8c2c9cd004e1e862e5a8ef85e9c9d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:50.706145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24d8c2c9cd004e1e862e5a8ef85e9c9d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:50.725435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24d8c2c9cd004e1e862e5a8ef85e9c9d'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:50.751910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1297/2600] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:50 2022] DELETE /identity/v3/users/24d8c2c9cd004e1e862e5a8ef85e9c9d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:50.752865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05514a14-ed31-4408-bae0-307e70a2d4c4 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:50.753157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05514a14-ed31-4408-bae0-307e70a2d4c4 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:50.753439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05514a14-ed31-4408-bae0-307e70a2d4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:50.778367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-05514a14-ed31-4408-bae0-307e70a2d4c4 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:50.778367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-05514a14-ed31-4408-bae0-307e70a2d4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:50.811718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=100 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:50.850909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1304/2601] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:50.867906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:50.869366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:50.939762 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca44174a-02b3-4de0-a6ef-4bc25b5332d2 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:50.943743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff29d87-9bd8-46b0-9548-4d6589ecfbc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:50.945143 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29d87-9bd8-46b0-9548-4d6589ecfbc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:50.945312 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29d87-9bd8-46b0-9548-4d6589ecfbc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:50.945551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff29d87-9bd8-46b0-9548-4d6589ecfbc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:50.955232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff29d87-9bd8-46b0-9548-4d6589ecfbc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:50.956730 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff29d87-9bd8-46b0-9548-4d6589ecfbc2 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:50.969399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1298/2602] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:51.040610 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05514a14-ed31-4408-bae0-307e70a2d4c4 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:51.042427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-05514a14-ed31-4408-bae0-307e70a2d4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:51.068204 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05514a14-ed31-4408-bae0-307e70a2d4c4 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:51.096776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:51.097753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:51.098034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:51.098304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] PATH_INFO: `/v3/users/ce83a48dd5874defb52933070d8a9b7e` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:51.099654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce83a48dd5874defb52933070d8a9b7e)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:51.101860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:51.188228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce83a48dd5874defb52933070d8a9b7e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:51.196618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce83a48dd5874defb52933070d8a9b7e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:51.198065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce83a48dd5874defb52933070d8a9b7e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:51.217510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce83a48dd5874defb52933070d8a9b7e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:51.222896 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce83a48dd5874defb52933070d8a9b7e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:51.252343 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1305/2603] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:51 2022] DELETE /identity/v3/users/ce83a48dd5874defb52933070d8a9b7e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:51.256726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a3ce7-0bc7-47fd-b41b-12ab4040754c None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:51.257014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a3ce7-0bc7-47fd-b41b-12ab4040754c None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:51.257297 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a3ce7-0bc7-47fd-b41b-12ab4040754c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:51.291190 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3e9a3ce7-0bc7-47fd-b41b-12ab4040754c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:51.292467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3e9a3ce7-0bc7-47fd-b41b-12ab4040754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:51.377394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1299/2604] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:51.417934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:51.419461 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:51.484694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2fe440-dcce-43db-bc7f-cac4d8c3553c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:51.488907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:51.489644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:51.489996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:51.490338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:51.501283 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:51.502742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:51.513295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1306/2605] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:51.904753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a3ce7-0bc7-47fd-b41b-12ab4040754c None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:51.906492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3e9a3ce7-0bc7-47fd-b41b-12ab4040754c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:51.923683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a3ce7-0bc7-47fd-b41b-12ab4040754c None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:51.932081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:51.933383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:51.933693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:51.934015 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] PATH_INFO: `/v3/projects/5e6157844f21442aac26dcb71092e6aa` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:51.935735 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6157844f21442aac26dcb71092e6aa)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:51.939785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:51.989393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:51.993506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:52.003773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:52.013306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23a2ac-e50e-44cd-9a0d-d380aea3ce84 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:52.034597 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6157844f21442aac26dcb71092e6aa', 'request_id': 'req-b06ff4c6-ce5c-4924-b5db-49042e494984'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.055999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6157844f21442aac26dcb71092e6aa', 'request_id': 'req-b06ff4c6-ce5c-4924-b5db-49042e494984'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.064251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6157844f21442aac26dcb71092e6aa', 'request_id': 'req-b06ff4c6-ce5c-4924-b5db-49042e494984'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.077905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] The token cache is being invalidate because project 5e6157844f21442aac26dcb71092e6aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:52.078452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.079306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.081034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1300/2606] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:51 2022] DELETE /identity/v3/projects/5e6157844f21442aac26dcb71092e6aa => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:52.085558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:52.087121 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:52.094279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:52.112904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:52.114379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:52.114764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:52.115140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] PATH_INFO: `/v3/projects/d6f560e71a42428ea4087c3e06350e88` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:52.116707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6f560e71a42428ea4087c3e06350e88)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:52.119753 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:52.164916 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06ff4c6-ce5c-4924-b5db-49042e494984 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:52.171020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:52.172114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:52.172395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:52.172667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] PATH_INFO: `/v3/users/5b65457e7b764bb19e94a18d404a9fcc` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:52.174216 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b65457e7b764bb19e94a18d404a9fcc)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:52.176713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:52.194745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6f560e71a42428ea4087c3e06350e88', 'request_id': 'req-4d146ffa-9c5c-43f8-9694-39af41fff2bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.205104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6f560e71a42428ea4087c3e06350e88', 'request_id': 'req-4d146ffa-9c5c-43f8-9694-39af41fff2bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.217647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6f560e71a42428ea4087c3e06350e88', 'request_id': 'req-4d146ffa-9c5c-43f8-9694-39af41fff2bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.238237 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] The token cache is being invalidate because project d6f560e71a42428ea4087c3e06350e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:52.238605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.239234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.245017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1307/2607] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:51 2022] DELETE /identity/v3/projects/d6f560e71a42428ea4087c3e06350e88 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:52.246231 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:52.248730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:52.253268 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b65457e7b764bb19e94a18d404a9fcc'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.254010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b65457e7b764bb19e94a18d404a9fcc'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.254906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b65457e7b764bb19e94a18d404a9fcc'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.267901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b65457e7b764bb19e94a18d404a9fcc'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.281251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b65457e7b764bb19e94a18d404a9fcc'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.308767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1301/2608] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:52 2022] DELETE /identity/v3/users/5b65457e7b764bb19e94a18d404a9fcc => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:52.337124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d146ffa-9c5c-43f8-9694-39af41fff2bb admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:52.412232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:52.413380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:52.413380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:52.414472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] PATH_INFO: `/v3/users/6b56a4151bec459da65e3fef1f66cbfb` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:52.415868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b56a4151bec459da65e3fef1f66cbfb)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:52.417833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:52.481954 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:52.525678 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b56a4151bec459da65e3fef1f66cbfb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.545866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b56a4151bec459da65e3fef1f66cbfb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.547197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b56a4151bec459da65e3fef1f66cbfb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.560568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b56a4151bec459da65e3fef1f66cbfb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.562162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b56a4151bec459da65e3fef1f66cbfb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.592410 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1308/2609] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:52 2022] DELETE /identity/v3/users/6b56a4151bec459da65e3fef1f66cbfb => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:52.681603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:52.683380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:52.702492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bf5532-db01-4ae5-a5b2-63282acb6623 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:52.749724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:52.799804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:52.802942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:52.803238 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:52.803557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] PATH_INFO: `/v3/projects/b490a608f24447a089fece717482333d` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:52.804896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b490a608f24447a089fece717482333d)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:52.806906 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:52.857918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b490a608f24447a089fece717482333d', 'request_id': 'req-f99cf8de-22ff-48a0-9db0-633aa2b55363'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.868203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b490a608f24447a089fece717482333d', 'request_id': 'req-f99cf8de-22ff-48a0-9db0-633aa2b55363'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.877408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b490a608f24447a089fece717482333d', 'request_id': 'req-f99cf8de-22ff-48a0-9db0-633aa2b55363'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.895060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] The token cache is being invalidate because project b490a608f24447a089fece717482333d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:52.895409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.896293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:52.898149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1302/2610] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:52 2022] DELETE /identity/v3/projects/b490a608f24447a089fece717482333d => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:52.902031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:52.903348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:52.924467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cb38cb-1822-482f-b2b7-aa4889003802 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:52.948993 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:52.950833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:53.041515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99cf8de-22ff-48a0-9db0-633aa2b55363 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:53.043319 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.044287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.044542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.045026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] PATH_INFO: `/v3/users/3baa07439f174053a18714bcec00b90d` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.046844 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3baa07439f174053a18714bcec00b90d)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.048184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.052086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.053532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.055003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.055648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] PATH_INFO: `/v3/projects/336bf74a2c1b4b7d8a83a1c194dbb9cb` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.058895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=336bf74a2c1b4b7d8a83a1c194dbb9cb)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.064146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.130645 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '336bf74a2c1b4b7d8a83a1c194dbb9cb', 'request_id': 'req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.142634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '336bf74a2c1b4b7d8a83a1c194dbb9cb', 'request_id': 'req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.159204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '336bf74a2c1b4b7d8a83a1c194dbb9cb', 'request_id': 'req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.161061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:53.166706 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3baa07439f174053a18714bcec00b90d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.173092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] The token cache is being invalidate because project 336bf74a2c1b4b7d8a83a1c194dbb9cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:53.173393 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.175249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6b74cf18-7bb2-4a2c-8c87-d91882be35a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.176420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3baa07439f174053a18714bcec00b90d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.177279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3baa07439f174053a18714bcec00b90d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.177534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1303/2611] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:52 2022] DELETE /identity/v3/projects/336bf74a2c1b4b7d8a83a1c194dbb9cb => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:53.182724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cf629-47fd-42b0-ad6d-d52ae6281c1f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.183202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cf629-47fd-42b0-ad6d-d52ae6281c1f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.183523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714cf629-47fd-42b0-ad6d-d52ae6281c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.188289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3baa07439f174053a18714bcec00b90d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.189572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9e7aecc1-7e3c-4d43-a362-95ba1ed7149b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3baa07439f174053a18714bcec00b90d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.203938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-714cf629-47fd-42b0-ad6d-d52ae6281c1f None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:53.205294 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-714cf629-47fd-42b0-ad6d-d52ae6281c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:53.208418 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1309/2612] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:52 2022] DELETE /identity/v3/users/3baa07439f174053a18714bcec00b90d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:53.284330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1304/2613] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:53.288085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f04ce-3f36-43c4-a045-8506e20ef6d3 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.288530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f04ce-3f36-43c4-a045-8506e20ef6d3 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.288975 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325f04ce-3f36-43c4-a045-8506e20ef6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.312853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-325f04ce-3f36-43c4-a045-8506e20ef6d3 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:53.314931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-325f04ce-3f36-43c4-a045-8506e20ef6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:53.326397 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1310/2614] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:53.331083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cf629-47fd-42b0-ad6d-d52ae6281c1f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:53.341263 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714cf629-47fd-42b0-ad6d-d52ae6281c1f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:53.346141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.346834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.347109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.347414 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.348152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:53.348458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.350089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.366034 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1305/2615] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:53.371056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f04ce-3f36-43c4-a045-8506e20ef6d3 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:53.381885 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325f04ce-3f36-43c4-a045-8506e20ef6d3 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:53.386911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76847b0-0b7c-486b-9fe1-1a757b228bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.387600 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76847b0-0b7c-486b-9fe1-1a757b228bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.387879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76847b0-0b7c-486b-9fe1-1a757b228bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.388152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76847b0-0b7c-486b-9fe1-1a757b228bd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.388898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76847b0-0b7c-486b-9fe1-1a757b228bd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.391331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76847b0-0b7c-486b-9fe1-1a757b228bd9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.416149 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:53.429398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1311/2616] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:53 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:53.429772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:53.442264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba7cffd-619c-4633-a308-1b6b96342ad9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:53.542100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c222-20da-47f1-ba43-c09b5238a2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.543800 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c222-20da-47f1-ba43-c09b5238a2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.544332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c222-20da-47f1-ba43-c09b5238a2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.544637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c222-20da-47f1-ba43-c09b5238a2ff admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.545636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7c222-20da-47f1-ba43-c09b5238a2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.547774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7c222-20da-47f1-ba43-c09b5238a2ff admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.582264 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1306/2617] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:53 2022] POST /identity/v3/users => generated 290 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:53.583630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76847b0-0b7c-486b-9fe1-1a757b228bd9 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:53.595808 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76847b0-0b7c-486b-9fe1-1a757b228bd9 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:53.601049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.601715 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.601905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.602184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.603193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.605050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.625327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1312/2618] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:53.626417 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c222-20da-47f1-ba43-c09b5238a2ff admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:53.631307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:53.634807 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:53.640492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c222-20da-47f1-ba43-c09b5238a2ff admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:53.648026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba223c66-b2bf-48d2-8e47-4cb1b27b7fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.648884 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba223c66-b2bf-48d2-8e47-4cb1b27b7fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.649099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba223c66-b2bf-48d2-8e47-4cb1b27b7fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.655488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba223c66-b2bf-48d2-8e47-4cb1b27b7fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.656617 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba223c66-b2bf-48d2-8e47-4cb1b27b7fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.659452 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba223c66-b2bf-48d2-8e47-4cb1b27b7fb0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.665157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f74188-42d6-487d-ab2b-604a54fcc2bd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:53.671112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1307/2619] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:53.674707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.674707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.674707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.674707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] PATH_INFO: `/v3/projects/082371c0e5ce4871918fa6a9ab5ca689` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.675099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] RBAC: Authorizing `identity:delete_project(project_id=082371c0e5ce4871918fa6a9ab5ca689)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.675398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba223c66-b2bf-48d2-8e47-4cb1b27b7fb0 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:53.676124 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.687861 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba223c66-b2bf-48d2-8e47-4cb1b27b7fb0 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:53.703231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608280ef-252a-4007-beac-be8ee88d331c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:53.704245 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608280ef-252a-4007-beac-be8ee88d331c admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.704538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608280ef-252a-4007-beac-be8ee88d331c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.704820 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608280ef-252a-4007-beac-be8ee88d331c admin admin] PATH_INFO: `/v3/projects/9f8b9c9e381f40fb8596f3cf9d6d1242/users/d2458e5718604e18a436860683d3755f/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.730237 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608280ef-252a-4007-beac-be8ee88d331c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f8b9c9e381f40fb8596f3cf9d6d1242, user_id=d2458e5718604e18a436860683d3755f, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:53.733863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608280ef-252a-4007-beac-be8ee88d331c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:53.754144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '082371c0e5ce4871918fa6a9ab5ca689', 'request_id': 'req-12d332a9-e13e-44fb-947b-059cdda2f49e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.756402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1308/2620] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:53 2022] PUT /identity/v3/projects/9f8b9c9e381f40fb8596f3cf9d6d1242/users/d2458e5718604e18a436860683d3755f/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:53.762141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '082371c0e5ce4871918fa6a9ab5ca689', 'request_id': 'req-12d332a9-e13e-44fb-947b-059cdda2f49e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.767942 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb0740-efe3-45d9-9376-d1d02ad8e3fa None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.769878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '082371c0e5ce4871918fa6a9ab5ca689', 'request_id': 'req-12d332a9-e13e-44fb-947b-059cdda2f49e'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.770249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb0740-efe3-45d9-9376-d1d02ad8e3fa None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.771105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb0740-efe3-45d9-9376-d1d02ad8e3fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.793493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] The token cache is being invalidate because project 082371c0e5ce4871918fa6a9ab5ca689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:53.793987 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.797977 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-12d332a9-e13e-44fb-947b-059cdda2f49e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:53.799308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1313/2621] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:53 2022] DELETE /identity/v3/projects/082371c0e5ce4871918fa6a9ab5ca689 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:53.803362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0e51-accd-4d16-89f3-6b5703e80761 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.803909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0e51-accd-4d16-89f3-6b5703e80761 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.805409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d0e51-accd-4d16-89f3-6b5703e80761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.832664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-310d0e51-accd-4d16-89f3-6b5703e80761 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:53.834243 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-310d0e51-accd-4d16-89f3-6b5703e80761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:53.836736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e0fb0740-efe3-45d9-9376-d1d02ad8e3fa None None] MFA Rules not processed for user `d2458e5718604e18a436860683d3755f`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:53.838661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e0fb0740-efe3-45d9-9376-d1d02ad8e3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:53.897032 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:53.922694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1314/2622] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:53.932057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261caf-9e7e-47b3-add4-d2fb5be8e2ee None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:53.932348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261caf-9e7e-47b3-add4-d2fb5be8e2ee None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:53.932632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0261caf-9e7e-47b3-add4-d2fb5be8e2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:53.974454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1309/2623] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:53 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:53.979409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb0740-efe3-45d9-9376-d1d02ad8e3fa None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:53.980833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e0fb0740-efe3-45d9-9376-d1d02ad8e3fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:53.986334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b0261caf-9e7e-47b3-add4-d2fb5be8e2ee None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:53.988094 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b0261caf-9e7e-47b3-add4-d2fb5be8e2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:53.997335 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb0740-efe3-45d9-9376-d1d02ad8e3fa None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.003008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a997e-686e-48d1-a59b-7ac721fc23dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.003743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a997e-686e-48d1-a59b-7ac721fc23dd admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.004024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a997e-686e-48d1-a59b-7ac721fc23dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.004289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a997e-686e-48d1-a59b-7ac721fc23dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.004996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a997e-686e-48d1-a59b-7ac721fc23dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.006982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04a997e-686e-48d1-a59b-7ac721fc23dd admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.008709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1315/2624] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:54.009685 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0261caf-9e7e-47b3-add4-d2fb5be8e2ee None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.022222 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0261caf-9e7e-47b3-add4-d2fb5be8e2ee None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.030680 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.031535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.031913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.032890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.034116 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:54.034524 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.037570 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.043946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1310/2625] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:53 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:20:54.045000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a997e-686e-48d1-a59b-7ac721fc23dd admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.054784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1316/2626] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:54.059155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.072126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8672c48-6035-4eb5-aec6-a70dcff74b78 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.122151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a997e-686e-48d1-a59b-7ac721fc23dd admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.127547 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28907613-ed2f-48a5-aa87-6e2ebb1b2701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.128256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28907613-ed2f-48a5-aa87-6e2ebb1b2701 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.128541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28907613-ed2f-48a5-aa87-6e2ebb1b2701 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.128823 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28907613-ed2f-48a5-aa87-6e2ebb1b2701 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.129525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28907613-ed2f-48a5-aa87-6e2ebb1b2701 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.131507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28907613-ed2f-48a5-aa87-6e2ebb1b2701 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.155212 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:54.173061 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064ffe-54c3-4f2f-a594-eb79e2de2f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.174827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064ffe-54c3-4f2f-a594-eb79e2de2f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.175109 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064ffe-54c3-4f2f-a594-eb79e2de2f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.176542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064ffe-54c3-4f2f-a594-eb79e2de2f04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.178344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1311/2627] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:54 2022] POST /identity/v3/users => generated 298 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:54.179045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064ffe-54c3-4f2f-a594-eb79e2de2f04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.183031 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28907613-ed2f-48a5-aa87-6e2ebb1b2701 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.183899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064ffe-54c3-4f2f-a594-eb79e2de2f04 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.209196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28907613-ed2f-48a5-aa87-6e2ebb1b2701 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.216818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c46283-d545-482f-b942-051d800a2c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.218695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46283-d545-482f-b942-051d800a2c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.219127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46283-d545-482f-b942-051d800a2c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.220083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c46283-d545-482f-b942-051d800a2c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.221113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c46283-d545-482f-b942-051d800a2c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.224635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c46283-d545-482f-b942-051d800a2c96 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.238968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1317/2628] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:54 2022] POST /identity/v3/projects => generated 397 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:54.242411 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064ffe-54c3-4f2f-a594-eb79e2de2f04 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.255776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1312/2629] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:54.259819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c46283-d545-482f-b942-051d800a2c96 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.273528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c46283-d545-482f-b942-051d800a2c96 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.340199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064ffe-54c3-4f2f-a594-eb79e2de2f04 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.347547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c639a-6c89-4025-b4ac-342419bd1e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.348395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c639a-6c89-4025-b4ac-342419bd1e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.348782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c639a-6c89-4025-b4ac-342419bd1e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.349175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5c639a-6c89-4025-b4ac-342419bd1e19 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.351417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c639a-6c89-4025-b4ac-342419bd1e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.354783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5c639a-6c89-4025-b4ac-342419bd1e19 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.370681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.372135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.373666 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.374099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] PATH_INFO: `/v3/projects/ece5973a70464af7bf9a7ed4e32a4994/users/fd64586da56b446facc5e926535f7e82/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.386052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1318/2630] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:54 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:54.387333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c639a-6c89-4025-b4ac-342419bd1e19 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.392801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece5973a70464af7bf9a7ed4e32a4994, user_id=fd64586da56b446facc5e926535f7e82, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.400413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5c639a-6c89-4025-b4ac-342419bd1e19 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.406102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1354cf7-f435-4a0e-bcfb-66d51edbd22f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.407036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1354cf7-f435-4a0e-bcfb-66d51edbd22f admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.407570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.407728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1354cf7-f435-4a0e-bcfb-66d51edbd22f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.408027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1354cf7-f435-4a0e-bcfb-66d51edbd22f admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.408992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1354cf7-f435-4a0e-bcfb-66d51edbd22f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.411532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1354cf7-f435-4a0e-bcfb-66d51edbd22f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.425235 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1319/2631] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:54.429380 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1354cf7-f435-4a0e-bcfb-66d51edbd22f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.446979 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1354cf7-f435-4a0e-bcfb-66d51edbd22f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.458723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1313/2632] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:54 2022] PUT /identity/v3/projects/ece5973a70464af7bf9a7ed4e32a4994/users/fd64586da56b446facc5e926535f7e82/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:54.459641 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.461723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:54.478486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:54.532368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362c9f1-999a-4c88-95b9-969d878427d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.533595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c9f1-999a-4c88-95b9-969d878427d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.533904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c9f1-999a-4c88-95b9-969d878427d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.534368 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7362c9f1-999a-4c88-95b9-969d878427d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.535119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362c9f1-999a-4c88-95b9-969d878427d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.537099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7362c9f1-999a-4c88-95b9-969d878427d3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.548687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d3a5-4f89-49eb-80e3-8077a15beefe admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.555296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1320/2633] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:54.556616 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362c9f1-999a-4c88-95b9-969d878427d3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.560656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.562413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.563551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.563844 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] PATH_INFO: `/v3/users/a7e574dc676841dea1f23a06c333ff14` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.566139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e574dc676841dea1f23a06c333ff14)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.568241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.572695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7362c9f1-999a-4c88-95b9-969d878427d3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.578266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d5538-ed57-4820-88f1-26bd5b7a9a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.578935 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5538-ed57-4820-88f1-26bd5b7a9a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.579216 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5538-ed57-4820-88f1-26bd5b7a9a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.579537 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850d5538-ed57-4820-88f1-26bd5b7a9a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.580395 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d5538-ed57-4820-88f1-26bd5b7a9a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.581978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850d5538-ed57-4820-88f1-26bd5b7a9a8f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.597510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1321/2634] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:54.597510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d5538-ed57-4820-88f1-26bd5b7a9a8f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.609985 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850d5538-ed57-4820-88f1-26bd5b7a9a8f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.614561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611267c-1aa7-4ec7-9dd5-2c72447538ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.615497 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611267c-1aa7-4ec7-9dd5-2c72447538ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.615771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611267c-1aa7-4ec7-9dd5-2c72447538ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.616037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a611267c-1aa7-4ec7-9dd5-2c72447538ba admin admin] PATH_INFO: `/v3/projects/bab48beabb5741edae35335d9fc8a4a9/users/4fa2293dab3a462bbec42df70b5058e0/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.633293 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611267c-1aa7-4ec7-9dd5-2c72447538ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab48beabb5741edae35335d9fc8a4a9, user_id=4fa2293dab3a462bbec42df70b5058e0, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.636245 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a611267c-1aa7-4ec7-9dd5-2c72447538ba admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.652308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1322/2635] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:54 2022] PUT /identity/v3/projects/bab48beabb5741edae35335d9fc8a4a9/users/4fa2293dab3a462bbec42df70b5058e0/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:54.653119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611267c-1aa7-4ec7-9dd5-2c72447538ba admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.671713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:54.710055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e574dc676841dea1f23a06c333ff14'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:54.713303 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e574dc676841dea1f23a06c333ff14'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:54.714756 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e574dc676841dea1f23a06c333ff14'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:54.728656 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e574dc676841dea1f23a06c333ff14'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:54.740814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-71f80448-b1d8-4a2c-875a-c367fbca48f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e574dc676841dea1f23a06c333ff14'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:54.751679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a611267c-1aa7-4ec7-9dd5-2c72447538ba admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.757265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af679342-0d09-4075-9e72-e3a058cf21f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.758304 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679342-0d09-4075-9e72-e3a058cf21f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.758592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679342-0d09-4075-9e72-e3a058cf21f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.759057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af679342-0d09-4075-9e72-e3a058cf21f4 admin admin] PATH_INFO: `/v3/projects/ece5973a70464af7bf9a7ed4e32a4994/users/fd64586da56b446facc5e926535f7e82/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.760635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1314/2636] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:54 2022] DELETE /identity/v3/users/a7e574dc676841dea1f23a06c333ff14 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:54.762113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9c232-f1d9-4642-93c5-a49466d08e31 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.762469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9c232-f1d9-4642-93c5-a49466d08e31 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.762819 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed9c232-f1d9-4642-93c5-a49466d08e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.764575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af679342-0d09-4075-9e72-e3a058cf21f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece5973a70464af7bf9a7ed4e32a4994, user_id=fd64586da56b446facc5e926535f7e82, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:54.769324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af679342-0d09-4075-9e72-e3a058cf21f4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:54.787806 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1323/2637] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:54 2022] PUT /identity/v3/projects/ece5973a70464af7bf9a7ed4e32a4994/users/fd64586da56b446facc5e926535f7e82/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:54.798454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3af2c-5f7d-41f0-b115-3afeeae3cf28 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.798742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3af2c-5f7d-41f0-b115-3afeeae3cf28 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.799016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3af2c-5f7d-41f0-b115-3afeeae3cf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:54.813946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-8ed9c232-f1d9-4642-93c5-a49466d08e31 None None] MFA Rules not processed for user `4fa2293dab3a462bbec42df70b5058e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:54.815521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8ed9c232-f1d9-4642-93c5-a49466d08e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:54.827435 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-21c3af2c-5f7d-41f0-b115-3afeeae3cf28 None None] MFA Rules not processed for user `fd64586da56b446facc5e926535f7e82`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:54.828649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-21c3af2c-5f7d-41f0-b115-3afeeae3cf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:54.898716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1315/2638] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:54.915411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9c232-f1d9-4642-93c5-a49466d08e31 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.917786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8ed9c232-f1d9-4642-93c5-a49466d08e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:54.920675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1324/2639] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:54.934913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3af2c-5f7d-41f0-b115-3afeeae3cf28 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:54.937309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-21c3af2c-5f7d-41f0-b115-3afeeae3cf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:54.945585 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:54.990741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed9c232-f1d9-4642-93c5-a49466d08e31 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:54.994767 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:54.995460 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:54.995746 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:54.996020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.004876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.006330 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.007781 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3af2c-5f7d-41f0-b115-3afeeae3cf28 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.011443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cff41d-fd85-4077-acd5-5cfc4db2c7b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.012092 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cff41d-fd85-4077-acd5-5cfc4db2c7b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.012375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cff41d-fd85-4077-acd5-5cfc4db2c7b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.012645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cff41d-fd85-4077-acd5-5cfc4db2c7b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.016180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1316/2640] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:54 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:55.016870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.018211 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.022279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cff41d-fd85-4077-acd5-5cfc4db2c7b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.023768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cff41d-fd85-4077-acd5-5cfc4db2c7b6 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.035729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1325/2641] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:20:55.100979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b97c1c-98ee-469c-90f6-b8b342383985 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.110382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.111431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.111648 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.111922 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] PATH_INFO: `/v3/projects/92bf8a2fea6243c28aff53d39a57f92c` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.123802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=92bf8a2fea6243c28aff53d39a57f92c)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.125780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.139153 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cff41d-fd85-4077-acd5-5cfc4db2c7b6 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.148211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cff41d-fd85-4077-acd5-5cfc4db2c7b6 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.152355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22aae0d7-04de-4390-b865-95e384917e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.153008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aae0d7-04de-4390-b865-95e384917e75 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.153292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aae0d7-04de-4390-b865-95e384917e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.153573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22aae0d7-04de-4390-b865-95e384917e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.155646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-22aae0d7-04de-4390-b865-95e384917e75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.195405 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92bf8a2fea6243c28aff53d39a57f92c', 'request_id': 'req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.205723 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92bf8a2fea6243c28aff53d39a57f92c', 'request_id': 'req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.209021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:55.213944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92bf8a2fea6243c28aff53d39a57f92c', 'request_id': 'req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.228620 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] The token cache is being invalidate because project 92bf8a2fea6243c28aff53d39a57f92c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:55.228946 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.229705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-a6b279cb-c01b-4e44-a9b2-7de1092be7ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.231891 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1317/2642] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:55 2022] DELETE /identity/v3/projects/92bf8a2fea6243c28aff53d39a57f92c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:55.236631 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d33d3-b721-4521-8aed-6c0936a18c84 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.237003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d33d3-b721-4521-8aed-6c0936a18c84 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.237131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aae0d7-04de-4390-b865-95e384917e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.237224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d33d3-b721-4521-8aed-6c0936a18c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.238496 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22aae0d7-04de-4390-b865-95e384917e75 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.250020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1326/2643] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:20:55.257691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3b0d33d3-b721-4521-8aed-6c0936a18c84 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:55.259061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3b0d33d3-b721-4521-8aed-6c0936a18c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.310247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:55.336288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1318/2644] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:55.344996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a624a-72a4-4545-9080-b84e5176e0f6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.345342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a624a-72a4-4545-9080-b84e5176e0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.345681 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13a624a-72a4-4545-9080-b84e5176e0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.367048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a13a624a-72a4-4545-9080-b84e5176e0f6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:55.368398 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a13a624a-72a4-4545-9080-b84e5176e0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.377577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1327/2645] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:55.381977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d33d3-b721-4521-8aed-6c0936a18c84 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.392890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d33d3-b721-4521-8aed-6c0936a18c84 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.397647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.398315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.398642 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.399068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.399876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:55.400189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.402278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.415319 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1319/2646] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:55.419973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a624a-72a4-4545-9080-b84e5176e0f6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.429572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13a624a-72a4-4545-9080-b84e5176e0f6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.434508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.435195 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.435508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.435798 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.436483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.438422 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.467427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1328/2647] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:55 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:55.473795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.487546 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cb6aea-0eb2-4851-b623-7624154efa3e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.566918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e5a1-2a1f-4593-a43e-e1fd8a2ef5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.567705 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e5a1-2a1f-4593-a43e-e1fd8a2ef5c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.567992 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e5a1-2a1f-4593-a43e-e1fd8a2ef5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.568276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1e5a1-2a1f-4593-a43e-e1fd8a2ef5c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.569035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1e5a1-2a1f-4593-a43e-e1fd8a2ef5c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.572540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1e5a1-2a1f-4593-a43e-e1fd8a2ef5c6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.573872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.575693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.591000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d289e5-5017-4810-bf6b-609f4e0f9c83 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.595304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1320/2648] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:55 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:55.596549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.597209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e5a1-2a1f-4593-a43e-e1fd8a2ef5c6 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.597543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.597861 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.598160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] PATH_INFO: `/v3/users/d2458e5718604e18a436860683d3755f` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.599538 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2458e5718604e18a436860683d3755f)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.601739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.607637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1e5a1-2a1f-4593-a43e-e1fd8a2ef5c6 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.612173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879f080-2092-47e6-9ddc-e47f68258d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.612830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879f080-2092-47e6-9ddc-e47f68258d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.613107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879f080-2092-47e6-9ddc-e47f68258d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.613376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4879f080-2092-47e6-9ddc-e47f68258d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.614098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879f080-2092-47e6-9ddc-e47f68258d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.615725 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4879f080-2092-47e6-9ddc-e47f68258d4b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.625804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1321/2649] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:55.630127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879f080-2092-47e6-9ddc-e47f68258d4b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.641789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4879f080-2092-47e6-9ddc-e47f68258d4b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.647123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c533b23-75af-4ef3-9846-7803549ce0af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.647933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c533b23-75af-4ef3-9846-7803549ce0af admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.648218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c533b23-75af-4ef3-9846-7803549ce0af admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.648543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c533b23-75af-4ef3-9846-7803549ce0af admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.649356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c533b23-75af-4ef3-9846-7803549ce0af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.651219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c533b23-75af-4ef3-9846-7803549ce0af admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.655048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2458e5718604e18a436860683d3755f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.662156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2458e5718604e18a436860683d3755f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.662467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1322/2650] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:55.663163 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2458e5718604e18a436860683d3755f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.666587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c533b23-75af-4ef3-9846-7803549ce0af admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.671927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2458e5718604e18a436860683d3755f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.672563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-68ec8abe-0a30-441f-925d-78b2db15ad48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2458e5718604e18a436860683d3755f'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:55.676365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c533b23-75af-4ef3-9846-7803549ce0af admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.681278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac5ac5-f694-4300-b9b0-d5fd7754dd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.682158 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5ac5-f694-4300-b9b0-d5fd7754dd99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.682439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5ac5-f694-4300-b9b0-d5fd7754dd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.682714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac5ac5-f694-4300-b9b0-d5fd7754dd99 admin admin] PATH_INFO: `/v3/projects/88c57cc932054d21a371617817d0b4b5/users/eb63369fbc5941588e6e2af9b35c0000/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.693941 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1329/2651] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:55 2022] DELETE /identity/v3/users/d2458e5718604e18a436860683d3755f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:55.695531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd706bdc-0048-4961-9386-9b21c79a39ab None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.695820 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd706bdc-0048-4961-9386-9b21c79a39ab None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.696117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd706bdc-0048-4961-9386-9b21c79a39ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.698512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac5ac5-f694-4300-b9b0-d5fd7754dd99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88c57cc932054d21a371617817d0b4b5, user_id=eb63369fbc5941588e6e2af9b35c0000, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.701276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac5ac5-f694-4300-b9b0-d5fd7754dd99 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.714211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-dd706bdc-0048-4961-9386-9b21c79a39ab None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:55.715449 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-dd706bdc-0048-4961-9386-9b21c79a39ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.716590 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1323/2652] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:55 2022] PUT /identity/v3/projects/88c57cc932054d21a371617817d0b4b5/users/eb63369fbc5941588e6e2af9b35c0000/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:55.727298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a59397-79ed-4200-93cd-9c9e420d8d37 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.727591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a59397-79ed-4200-93cd-9c9e420d8d37 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.727868 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a59397-79ed-4200-93cd-9c9e420d8d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.753824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-01a59397-79ed-4200-93cd-9c9e420d8d37 None None] MFA Rules not processed for user `eb63369fbc5941588e6e2af9b35c0000`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:55.755115 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-01a59397-79ed-4200-93cd-9c9e420d8d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.791049 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1330/2653] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:55.800962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd706bdc-0048-4961-9386-9b21c79a39ab None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.802881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-dd706bdc-0048-4961-9386-9b21c79a39ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.838990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:55.848990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1324/2654] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:55.863122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a59397-79ed-4200-93cd-9c9e420d8d37 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:55.868142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-01a59397-79ed-4200-93cd-9c9e420d8d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:55.880910 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:55.898541 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd706bdc-0048-4961-9386-9b21c79a39ab None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.902372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.903091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.903521 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.903872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.914315 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.916067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:55.927314 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1331/2655] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:20:55.972537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a59397-79ed-4200-93cd-9c9e420d8d37 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:55.979496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfc583-9bb3-4ed9-b16b-d9a6a4f72ce4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:55.980159 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfc583-9bb3-4ed9-b16b-d9a6a4f72ce4 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:55.980438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfc583-9bb3-4ed9-b16b-d9a6a4f72ce4 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:55.980712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bfc583-9bb3-4ed9-b16b-d9a6a4f72ce4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:55.989186 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfc583-9bb3-4ed9-b16b-d9a6a4f72ce4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:55.990818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bfc583-9bb3-4ed9-b16b-d9a6a4f72ce4 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:56.001083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1325/2656] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:56.259180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:56.260717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:56.276960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c378d4e4-b307-4078-8797-ee27206d0bb2 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:56.287120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:56.288108 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:56.288425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:56.288815 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] PATH_INFO: `/v3/projects/9f8b9c9e381f40fb8596f3cf9d6d1242` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:56.290239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f8b9c9e381f40fb8596f3cf9d6d1242)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:56.292531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:56.346430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f8b9c9e381f40fb8596f3cf9d6d1242', 'request_id': 'req-b3e84e8d-618a-444e-ace6-c74d656dc62d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.352842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f8b9c9e381f40fb8596f3cf9d6d1242', 'request_id': 'req-b3e84e8d-618a-444e-ace6-c74d656dc62d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.359591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f8b9c9e381f40fb8596f3cf9d6d1242', 'request_id': 'req-b3e84e8d-618a-444e-ace6-c74d656dc62d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.375119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] The token cache is being invalidate because project 9f8b9c9e381f40fb8596f3cf9d6d1242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:56.375460 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.377622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.379717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1332/2657] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:56 2022] DELETE /identity/v3/projects/9f8b9c9e381f40fb8596f3cf9d6d1242 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:56.384214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfc583-9bb3-4ed9-b16b-d9a6a4f72ce4 service nova] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:56.397849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bfc583-9bb3-4ed9-b16b-d9a6a4f72ce4 service nova] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:56.459251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:56.460157 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:56.460436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:56.460703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] PATH_INFO: `/v3/users/fd64586da56b446facc5e926535f7e82` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:56.461878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd64586da56b446facc5e926535f7e82)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:56.463841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:56.514902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd64586da56b446facc5e926535f7e82'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.515977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd64586da56b446facc5e926535f7e82'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.516677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd64586da56b446facc5e926535f7e82'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.526254 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd64586da56b446facc5e926535f7e82'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.534093 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-ef0c2945-fa7b-48ea-b4e6-dae66d528642 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd64586da56b446facc5e926535f7e82'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.556039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1326/2658] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:20:56 2022] DELETE /identity/v3/users/fd64586da56b446facc5e926535f7e82 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:56.848367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:56.849606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:56.863494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e84e8d-618a-444e-ace6-c74d656dc62d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:56.921747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:56.925361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:56.925654 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:56.925932 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] PATH_INFO: `/v3/projects/ece5973a70464af7bf9a7ed4e32a4994` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:56.931403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece5973a70464af7bf9a7ed4e32a4994)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:56.935103 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:56.984571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:56.986110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece5973a70464af7bf9a7ed4e32a4994', 'request_id': 'req-b816cc92-1f63-42ad-8980-e18132e75296'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.992939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece5973a70464af7bf9a7ed4e32a4994', 'request_id': 'req-b816cc92-1f63-42ad-8980-e18132e75296'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:56.999039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece5973a70464af7bf9a7ed4e32a4994', 'request_id': 'req-b816cc92-1f63-42ad-8980-e18132e75296'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:57.012026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] The token cache is being invalidate because project ece5973a70464af7bf9a7ed4e32a4994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:20:57.012308 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:57.012914 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b816cc92-1f63-42ad-8980-e18132e75296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:20:57.014554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1333/2659] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:20:56 2022] DELETE /identity/v3/projects/ece5973a70464af7bf9a7ed4e32a4994 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:57.020205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a98d6-367d-4ce1-a29b-dc0cca162b50 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.020677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a98d6-367d-4ce1-a29b-dc0cca162b50 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.020895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a98d6-367d-4ce1-a29b-dc0cca162b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.040753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0f9a98d6-367d-4ce1-a29b-dc0cca162b50 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:57.042056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0f9a98d6-367d-4ce1-a29b-dc0cca162b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:57.052741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:57.105448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1327/2660] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:20:57.114141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bff9c-4ddb-4afc-90bd-92b9dc14ab24 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.114431 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bff9c-4ddb-4afc-90bd-92b9dc14ab24 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.114746 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5bff9c-4ddb-4afc-90bd-92b9dc14ab24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.133256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-bc5bff9c-4ddb-4afc-90bd-92b9dc14ab24 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:57.134475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-bc5bff9c-4ddb-4afc-90bd-92b9dc14ab24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:57.142750 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1334/2661] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:57.147037 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a98d6-367d-4ce1-a29b-dc0cca162b50 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:57.156978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a98d6-367d-4ce1-a29b-dc0cca162b50 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:57.162189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:57.162676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.162961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.163236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.164004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:20:57.164317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:57.166009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:57.181003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1328/2662] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:20:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:20:57.185296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bff9c-4ddb-4afc-90bd-92b9dc14ab24 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:57.195188 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5bff9c-4ddb-4afc-90bd-92b9dc14ab24 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:57.200709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead9a63-2e00-4e6b-b8f4-04253d1fdddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:57.201474 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9a63-2e00-4e6b-b8f4-04253d1fdddc admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.201756 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9a63-2e00-4e6b-b8f4-04253d1fdddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.202029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eead9a63-2e00-4e6b-b8f4-04253d1fdddc admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.202730 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead9a63-2e00-4e6b-b8f4-04253d1fdddc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:57.204682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eead9a63-2e00-4e6b-b8f4-04253d1fdddc admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:57.233737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1335/2663] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:57 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:57.237826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:57.247740 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893c205e-b0c3-45e4-be19-1ddd9deb3695 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:57.303056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a2c0-d77f-4d02-84db-13567768d325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:57.303759 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a2c0-d77f-4d02-84db-13567768d325 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.304043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a2c0-d77f-4d02-84db-13567768d325 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.304317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a2c0-d77f-4d02-84db-13567768d325 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.305022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2a2c0-d77f-4d02-84db-13567768d325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:57.306945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a2a2c0-d77f-4d02-84db-13567768d325 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:57.329241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1329/2664] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:57 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:57.333420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead9a63-2e00-4e6b-b8f4-04253d1fdddc admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:57.344034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eead9a63-2e00-4e6b-b8f4-04253d1fdddc admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:57.349050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36159ea4-b014-40ca-b215-a7ba9fce8779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:57.349712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36159ea4-b014-40ca-b215-a7ba9fce8779 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.349989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36159ea4-b014-40ca-b215-a7ba9fce8779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.350258 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36159ea4-b014-40ca-b215-a7ba9fce8779 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.351025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36159ea4-b014-40ca-b215-a7ba9fce8779 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:57.352760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36159ea4-b014-40ca-b215-a7ba9fce8779 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:57.363553 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1336/2665] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:57.367535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a2c0-d77f-4d02-84db-13567768d325 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:57.378080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a2c0-d77f-4d02-84db-13567768d325 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:57.384056 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57908e-5532-40c5-8687-c927b2fff805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:57.384721 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57908e-5532-40c5-8687-c927b2fff805 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.385010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57908e-5532-40c5-8687-c927b2fff805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.385284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57908e-5532-40c5-8687-c927b2fff805 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.386020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57908e-5532-40c5-8687-c927b2fff805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:57.387684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57908e-5532-40c5-8687-c927b2fff805 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:57.398708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1330/2666] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:57.402867 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36159ea4-b014-40ca-b215-a7ba9fce8779 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:57.413351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36159ea4-b014-40ca-b215-a7ba9fce8779 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:57.419120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:57.420046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.420330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.420602 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] PATH_INFO: `/v3/projects/aac6a376528d47d5a00b91de9b2df9d0/users/27489c3dc91649eba176fb290f3e96a3/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.437228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac6a376528d47d5a00b91de9b2df9d0, user_id=27489c3dc91649eba176fb290f3e96a3, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:57.439899 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:57.457476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1337/2667] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:57 2022] PUT /identity/v3/projects/aac6a376528d47d5a00b91de9b2df9d0/users/27489c3dc91649eba176fb290f3e96a3/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:57.462773 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d89e-b08c-44ba-9128-704680f5b17b None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.463224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d89e-b08c-44ba-9128-704680f5b17b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.463531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f719d89e-b08c-44ba-9128-704680f5b17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.482362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f719d89e-b08c-44ba-9128-704680f5b17b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:57.483647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f719d89e-b08c-44ba-9128-704680f5b17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:57.548171 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1331/2668] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:57.558317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:57.560192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:57.617120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8477023-2731-46db-a1a1-66ce54a33029 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:57.621056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70170e9-f69e-4e5f-9610-b864e48cd81f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:57.621716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70170e9-f69e-4e5f-9610-b864e48cd81f service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:57.622056 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70170e9-f69e-4e5f-9610-b864e48cd81f service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:57.622333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70170e9-f69e-4e5f-9610-b864e48cd81f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:57.631770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70170e9-f69e-4e5f-9610-b864e48cd81f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:57.633279 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70170e9-f69e-4e5f-9610-b864e48cd81f service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:57.644254 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1338/2669] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:20:58.359597 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521599f1-9d09-478c-b374-e9c028c56d9a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:58.360394 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521599f1-9d09-478c-b374-e9c028c56d9a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:58.360758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521599f1-9d09-478c-b374-e9c028c56d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:58.391520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-521599f1-9d09-478c-b374-e9c028c56d9a None None] MFA Rules not processed for user `27489c3dc91649eba176fb290f3e96a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:58.392743 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-521599f1-9d09-478c-b374-e9c028c56d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:58.401012 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:58.458860 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1332/2670] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:58.463812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70170e9-f69e-4e5f-9610-b864e48cd81f service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:58.473925 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70170e9-f69e-4e5f-9610-b864e48cd81f service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:58.479009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5b8ce-3bb6-4c2e-9d8b-a57c8678c5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:58.479701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5b8ce-3bb6-4c2e-9d8b-a57c8678c5ec admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:58.479988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5b8ce-3bb6-4c2e-9d8b-a57c8678c5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:58.480250 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5b8ce-3bb6-4c2e-9d8b-a57c8678c5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:58.480947 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5b8ce-3bb6-4c2e-9d8b-a57c8678c5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:58.482956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d5b8ce-3bb6-4c2e-9d8b-a57c8678c5ec admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:58.514646 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1339/2671] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:20:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:58.516380 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521599f1-9d09-478c-b374-e9c028c56d9a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:58.527363 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521599f1-9d09-478c-b374-e9c028c56d9a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:58.584322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626b059-fd2f-4662-be63-fb9df76d1a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:58.585026 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b059-fd2f-4662-be63-fb9df76d1a03 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:58.585307 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b059-fd2f-4662-be63-fb9df76d1a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:58.585581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b059-fd2f-4662-be63-fb9df76d1a03 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:58.586276 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626b059-fd2f-4662-be63-fb9df76d1a03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:58.588255 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f626b059-fd2f-4662-be63-fb9df76d1a03 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:58.610395 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1333/2672] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:20:58 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:20:58.614518 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5b8ce-3bb6-4c2e-9d8b-a57c8678c5ec admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:58.625412 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5b8ce-3bb6-4c2e-9d8b-a57c8678c5ec admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:58.630984 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e52654-0221-4a22-8704-49e484312f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:58.631677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e52654-0221-4a22-8704-49e484312f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:58.631980 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e52654-0221-4a22-8704-49e484312f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:58.632261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e52654-0221-4a22-8704-49e484312f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:58.632995 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e52654-0221-4a22-8704-49e484312f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:58.634658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e52654-0221-4a22-8704-49e484312f8c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:58.647375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1340/2673] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 20 09:20:58.651567 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626b059-fd2f-4662-be63-fb9df76d1a03 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:58.661544 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f626b059-fd2f-4662-be63-fb9df76d1a03 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:58.666084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa03e5-aead-4d47-a408-75baaad852d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:58.666969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa03e5-aead-4d47-a408-75baaad852d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:58.667300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa03e5-aead-4d47-a408-75baaad852d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:58.667593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa03e5-aead-4d47-a408-75baaad852d6 admin admin] PATH_INFO: `/v3/projects/70a920cc4a634d119a0f61f4bbc567e4/users/114ed6a689514c058f36ef08e3cb9f0b/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:58.682670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa03e5-aead-4d47-a408-75baaad852d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a920cc4a634d119a0f61f4bbc567e4, user_id=114ed6a689514c058f36ef08e3cb9f0b, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:58.685324 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa03e5-aead-4d47-a408-75baaad852d6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:58.700948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1334/2674] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:58 2022] PUT /identity/v3/projects/70a920cc4a634d119a0f61f4bbc567e4/users/114ed6a689514c058f36ef08e3cb9f0b/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:20:58.705157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e52654-0221-4a22-8704-49e484312f8c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:58.715389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e52654-0221-4a22-8704-49e484312f8c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:58.756262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:58.779717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7a6a1-0cbc-4f2c-ae76-6c01eb84240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:58.780533 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7a6a1-0cbc-4f2c-ae76-6c01eb84240e admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:58.780821 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7a6a1-0cbc-4f2c-ae76-6c01eb84240e admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:58.781100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7a6a1-0cbc-4f2c-ae76-6c01eb84240e admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:58.781843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7a6a1-0cbc-4f2c-ae76-6c01eb84240e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:58.783546 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc7a6a1-0cbc-4f2c-ae76-6c01eb84240e admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:58.793508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1341/2675] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:20:58.797732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa03e5-aead-4d47-a408-75baaad852d6 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:58.808193 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa03e5-aead-4d47-a408-75baaad852d6 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:58.812981 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:58.813866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:58.814151 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:58.814429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] PATH_INFO: `/v3/projects/70a920cc4a634d119a0f61f4bbc567e4/users/114ed6a689514c058f36ef08e3cb9f0b/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:58.816476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70a920cc4a634d119a0f61f4bbc567e4, user_id=114ed6a689514c058f36ef08e3cb9f0b, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:58.819114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:58.829754 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1335/2676] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:20:58 2022] PUT /identity/v3/projects/70a920cc4a634d119a0f61f4bbc567e4/users/114ed6a689514c058f36ef08e3cb9f0b/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:20:59.471774 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7163818-2d22-469a-b1c8-f3e1888048cc None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:59.472428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7163818-2d22-469a-b1c8-f3e1888048cc None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:59.472428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7163818-2d22-469a-b1c8-f3e1888048cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:59.505287 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-d7163818-2d22-469a-b1c8-f3e1888048cc None None] MFA Rules not processed for user `114ed6a689514c058f36ef08e3cb9f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:20:59.506726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d7163818-2d22-469a-b1c8-f3e1888048cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:59.593053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1342/2677] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:20:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:20:59.604714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:20:59.606444 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:20:59.665511 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d7ac4d-e218-4644-9d16-e62e4bfda9d0 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:20:59.669065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12fd23b-549d-4f06-ad89-d4a101702e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:20:59.669733 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fd23b-549d-4f06-ad89-d4a101702e44 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:20:59.670014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fd23b-549d-4f06-ad89-d4a101702e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:20:59.670288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12fd23b-549d-4f06-ad89-d4a101702e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:20:59.682424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:20:59.734975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fd23b-549d-4f06-ad89-d4a101702e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:20:59.736435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12fd23b-549d-4f06-ad89-d4a101702e44 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:20:59.745804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1336/2678] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:20:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:00.874910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7163818-2d22-469a-b1c8-f3e1888048cc None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:00.876372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d7163818-2d22-469a-b1c8-f3e1888048cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:00.965494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7163818-2d22-469a-b1c8-f3e1888048cc None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:00.970733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:00.971670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:00.971901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:00.972192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] PATH_INFO: `/v3/users/eb63369fbc5941588e6e2af9b35c0000` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:00.973362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb63369fbc5941588e6e2af9b35c0000)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:00.980213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:01.046575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb63369fbc5941588e6e2af9b35c0000'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.059171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb63369fbc5941588e6e2af9b35c0000'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.060065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb63369fbc5941588e6e2af9b35c0000'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.071076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb63369fbc5941588e6e2af9b35c0000'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.071876 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb63369fbc5941588e6e2af9b35c0000'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.077776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:01.100483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1343/2679] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:00 2022] DELETE /identity/v3/users/eb63369fbc5941588e6e2af9b35c0000 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:01.105267 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b180d-3b51-4dfb-8dbe-693750bb84cf None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.105719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b180d-3b51-4dfb-8dbe-693750bb84cf None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.105843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b180d-3b51-4dfb-8dbe-693750bb84cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.110096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:01.111764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:01.131696 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-af2b180d-3b51-4dfb-8dbe-693750bb84cf None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:01.133028 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-af2b180d-3b51-4dfb-8dbe-693750bb84cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:01.183420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b69b1c-bb17-469a-b886-80626581a6db admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:01.187182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:01.187933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.188337 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.188682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.190860 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:01.213608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1337/2680] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:01.225892 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b180d-3b51-4dfb-8dbe-693750bb84cf None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:01.226996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-af2b180d-3b51-4dfb-8dbe-693750bb84cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:01.261567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:01.263091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:01.273466 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1344/2681] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:21:01.286513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b180d-3b51-4dfb-8dbe-693750bb84cf None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:01.290075 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08e6385-5ea7-42ec-9871-9b6dc47b871b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:01.290843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e6385-5ea7-42ec-9871-9b6dc47b871b service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.291129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e6385-5ea7-42ec-9871-9b6dc47b871b service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.291443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08e6385-5ea7-42ec-9871-9b6dc47b871b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.302087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e6385-5ea7-42ec-9871-9b6dc47b871b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:01.303709 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08e6385-5ea7-42ec-9871-9b6dc47b871b service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:01.313467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1338/2682] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 20 09:21:01.588635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:01.590724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:01.610965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5b9051-e620-4e13-b038-0c7791002366 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:01.616363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:01.617213 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.617486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.617757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] PATH_INFO: `/v3/projects/88c57cc932054d21a371617817d0b4b5` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.619065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] RBAC: Authorizing `identity:delete_project(project_id=88c57cc932054d21a371617817d0b4b5)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:01.621074 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:01.672718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88c57cc932054d21a371617817d0b4b5', 'request_id': 'req-2f486b94-2590-4a46-9ac5-1cb04ea4957c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.678479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88c57cc932054d21a371617817d0b4b5', 'request_id': 'req-2f486b94-2590-4a46-9ac5-1cb04ea4957c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.686264 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88c57cc932054d21a371617817d0b4b5', 'request_id': 'req-2f486b94-2590-4a46-9ac5-1cb04ea4957c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.701694 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] The token cache is being invalidate because project 88c57cc932054d21a371617817d0b4b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:01.701983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.702593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f486b94-2590-4a46-9ac5-1cb04ea4957c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:01.704228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1345/2683] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:01 2022] DELETE /identity/v3/projects/88c57cc932054d21a371617817d0b4b5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:01.709220 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bebf82-cc7d-428f-ac36-d16a9dbb811b None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.709512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bebf82-cc7d-428f-ac36-d16a9dbb811b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.709821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bebf82-cc7d-428f-ac36-d16a9dbb811b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.728013 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-31bebf82-cc7d-428f-ac36-d16a9dbb811b None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:01.729242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-31bebf82-cc7d-428f-ac36-d16a9dbb811b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:01.782463 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:01.791691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1339/2684] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:01.799499 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3ab7c-d3f8-44c5-bc96-b6b392dbec48 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.799793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3ab7c-d3f8-44c5-bc96-b6b392dbec48 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.800073 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a3ab7c-d3f8-44c5-bc96-b6b392dbec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.818219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-28a3ab7c-d3f8-44c5-bc96-b6b392dbec48 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:01.819451 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-28a3ab7c-d3f8-44c5-bc96-b6b392dbec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:01.827565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1346/2685] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:01.831681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bebf82-cc7d-428f-ac36-d16a9dbb811b None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:01.841484 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bebf82-cc7d-428f-ac36-d16a9dbb811b None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:01.847248 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:01.847941 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.848221 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.848493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.849224 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:21:01.849535 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:01.851189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:01.863388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1340/2686] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:21:01.867587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3ab7c-d3f8-44c5-bc96-b6b392dbec48 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:01.877085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a3ab7c-d3f8-44c5-bc96-b6b392dbec48 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:01.882909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9648d9a-63e3-418f-9ff3-96590e1a2bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:01.883660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9648d9a-63e3-418f-9ff3-96590e1a2bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.883927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9648d9a-63e3-418f-9ff3-96590e1a2bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.884191 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9648d9a-63e3-418f-9ff3-96590e1a2bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.884877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9648d9a-63e3-418f-9ff3-96590e1a2bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:01.886787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9648d9a-63e3-418f-9ff3-96590e1a2bdf admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:01.891598 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:01.919604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1347/2687] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:01 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:01.924079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:01.934549 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c724980e-57c2-4059-b857-4cacae34fcec admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:01.992678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5713a52-f183-48b7-8a64-77b0d4df45c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:01.993355 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713a52-f183-48b7-8a64-77b0d4df45c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:01.993624 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713a52-f183-48b7-8a64-77b0d4df45c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:01.993900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5713a52-f183-48b7-8a64-77b0d4df45c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:01.994576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5713a52-f183-48b7-8a64-77b0d4df45c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:01.996552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5713a52-f183-48b7-8a64-77b0d4df45c6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.021998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1341/2688] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:01 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:02.023950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9648d9a-63e3-418f-9ff3-96590e1a2bdf admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.034393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9648d9a-63e3-418f-9ff3-96590e1a2bdf admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.039200 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dab82-f7eb-4fad-84ad-0e2eb6e82794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.039888 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dab82-f7eb-4fad-84ad-0e2eb6e82794 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.040170 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dab82-f7eb-4fad-84ad-0e2eb6e82794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.040446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958dab82-f7eb-4fad-84ad-0e2eb6e82794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.041176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dab82-f7eb-4fad-84ad-0e2eb6e82794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.042768 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958dab82-f7eb-4fad-84ad-0e2eb6e82794 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.052606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1348/2689] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:02.056456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5713a52-f183-48b7-8a64-77b0d4df45c6 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.067795 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5713a52-f183-48b7-8a64-77b0d4df45c6 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.073473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ce4ea-507d-4e39-942e-aa8dbebfeec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.074169 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ce4ea-507d-4e39-942e-aa8dbebfeec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.074458 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ce4ea-507d-4e39-942e-aa8dbebfeec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.074728 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ce4ea-507d-4e39-942e-aa8dbebfeec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.075503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ce4ea-507d-4e39-942e-aa8dbebfeec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.077115 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ce4ea-507d-4e39-942e-aa8dbebfeec9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.091781 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1342/2690] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:02.092329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dab82-f7eb-4fad-84ad-0e2eb6e82794 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.102833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958dab82-f7eb-4fad-84ad-0e2eb6e82794 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.108475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6ccf4-10f5-4d98-9833-fba122f2d3be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.109577 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6ccf4-10f5-4d98-9833-fba122f2d3be admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.109863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6ccf4-10f5-4d98-9833-fba122f2d3be admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.110134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6ccf4-10f5-4d98-9833-fba122f2d3be admin admin] PATH_INFO: `/v3/projects/7977fb300d884296be5a96d690bfb610/users/e8058cfa8aa444208892078ae17abe0f/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.126758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6ccf4-10f5-4d98-9833-fba122f2d3be admin admin] RBAC: Authorizing `identity:create_grant(project_id=7977fb300d884296be5a96d690bfb610, user_id=e8058cfa8aa444208892078ae17abe0f, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.129433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6ccf4-10f5-4d98-9833-fba122f2d3be admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.153296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1349/2691] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:02 2022] PUT /identity/v3/projects/7977fb300d884296be5a96d690bfb610/users/e8058cfa8aa444208892078ae17abe0f/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:02.164334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5c076-e0f2-463e-a943-b9abd8b5d2d7 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.164900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5c076-e0f2-463e-a943-b9abd8b5d2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.164900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5c076-e0f2-463e-a943-b9abd8b5d2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.193708 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-aaf5c076-e0f2-463e-a943-b9abd8b5d2d7 None None] MFA Rules not processed for user `e8058cfa8aa444208892078ae17abe0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:02.195429 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-aaf5c076-e0f2-463e-a943-b9abd8b5d2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:02.260695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1343/2692] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:02.265295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6ccf4-10f5-4d98-9833-fba122f2d3be admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.328782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6ccf4-10f5-4d98-9833-fba122f2d3be admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.333850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3a432-f8d6-4bf3-a30e-8fa7a1769834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.334476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3a432-f8d6-4bf3-a30e-8fa7a1769834 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.334788 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3a432-f8d6-4bf3-a30e-8fa7a1769834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.335102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c3a432-f8d6-4bf3-a30e-8fa7a1769834 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.335814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3a432-f8d6-4bf3-a30e-8fa7a1769834 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.337711 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c3a432-f8d6-4bf3-a30e-8fa7a1769834 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.363901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1350/2693] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:02 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:21:02.368044 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5c076-e0f2-463e-a943-b9abd8b5d2d7 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.379724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5c076-e0f2-463e-a943-b9abd8b5d2d7 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.434958 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:02.443442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a6606-5315-453a-aee7-73fca49be99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.444123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a6606-5315-453a-aee7-73fca49be99e admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.444408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a6606-5315-453a-aee7-73fca49be99e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.444684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a6606-5315-453a-aee7-73fca49be99e admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.445381 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a6606-5315-453a-aee7-73fca49be99e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.447376 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a6606-5315-453a-aee7-73fca49be99e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.470472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1344/2694] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:02 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:02.474630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3a432-f8d6-4bf3-a30e-8fa7a1769834 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.485770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c3a432-f8d6-4bf3-a30e-8fa7a1769834 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.490851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609b026-1580-4dd2-991c-0f353cb472bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.491609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609b026-1580-4dd2-991c-0f353cb472bd admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.491894 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609b026-1580-4dd2-991c-0f353cb472bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.492187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609b026-1580-4dd2-991c-0f353cb472bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.492916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609b026-1580-4dd2-991c-0f353cb472bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.494579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4609b026-1580-4dd2-991c-0f353cb472bd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.510353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1351/2695] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:02.511805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a6606-5315-453a-aee7-73fca49be99e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.526799 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a6606-5315-453a-aee7-73fca49be99e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.528403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2954f-d54b-40b2-a03e-38f94841c5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.529498 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2954f-d54b-40b2-a03e-38f94841c5c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.529785 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2954f-d54b-40b2-a03e-38f94841c5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.530064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f2954f-d54b-40b2-a03e-38f94841c5c9 admin admin] PATH_INFO: `/v3/projects/505c18e377e94b7db16b77c568d53e99/users/de6a2580af1c4eb99f841adcc83496ce/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.546410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2954f-d54b-40b2-a03e-38f94841c5c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=505c18e377e94b7db16b77c568d53e99, user_id=de6a2580af1c4eb99f841adcc83496ce, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.549126 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f2954f-d54b-40b2-a03e-38f94841c5c9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.563520 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1345/2696] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:02 2022] PUT /identity/v3/projects/505c18e377e94b7db16b77c568d53e99/users/de6a2580af1c4eb99f841adcc83496ce/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:02.567555 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609b026-1580-4dd2-991c-0f353cb472bd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.577567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609b026-1580-4dd2-991c-0f353cb472bd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.634663 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478e6ff-3f76-45dd-bac6-0486e1944ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.635331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478e6ff-3f76-45dd-bac6-0486e1944ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.635970 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478e6ff-3f76-45dd-bac6-0486e1944ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.636087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478e6ff-3f76-45dd-bac6-0486e1944ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.636912 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478e6ff-3f76-45dd-bac6-0486e1944ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.638816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5478e6ff-3f76-45dd-bac6-0486e1944ffb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.651488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1352/2697] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:02.656048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2954f-d54b-40b2-a03e-38f94841c5c9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.667632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f2954f-d54b-40b2-a03e-38f94841c5c9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.672639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.673660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.673990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.674300 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] PATH_INFO: `/v3/projects/505c18e377e94b7db16b77c568d53e99/users/de6a2580af1c4eb99f841adcc83496ce/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.676421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] RBAC: Authorizing `identity:create_grant(project_id=505c18e377e94b7db16b77c568d53e99, user_id=de6a2580af1c4eb99f841adcc83496ce, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.679309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.690172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1346/2698] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:02 2022] PUT /identity/v3/projects/505c18e377e94b7db16b77c568d53e99/users/de6a2580af1c4eb99f841adcc83496ce/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:02.702948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf391ba5-9468-4186-9fa3-bcdbcef21f80 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.703309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf391ba5-9468-4186-9fa3-bcdbcef21f80 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.703606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf391ba5-9468-4186-9fa3-bcdbcef21f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.732370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-bf391ba5-9468-4186-9fa3-bcdbcef21f80 None None] MFA Rules not processed for user `de6a2580af1c4eb99f841adcc83496ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:02.733672 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-bf391ba5-9468-4186-9fa3-bcdbcef21f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:02.743201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:02.819878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1353/2699] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:02 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:02.832106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:02.833971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:02.894070 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55cba9-c51d-4e76-a529-661e2507d936 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:02.898320 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6fafb-2d25-4aee-9c32-d431ed4b9053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:02.898977 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6fafb-2d25-4aee-9c32-d431ed4b9053 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:02.899275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6fafb-2d25-4aee-9c32-d431ed4b9053 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:02.899564 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6fafb-2d25-4aee-9c32-d431ed4b9053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:02.908980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6fafb-2d25-4aee-9c32-d431ed4b9053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:02.910431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6fafb-2d25-4aee-9c32-d431ed4b9053 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:02.919830 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1347/2700] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:02 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:03.266425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf391ba5-9468-4186-9fa3-bcdbcef21f80 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:03.268060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-bf391ba5-9468-4186-9fa3-bcdbcef21f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:03.337500 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf391ba5-9468-4186-9fa3-bcdbcef21f80 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:03.345332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6fafb-2d25-4aee-9c32-d431ed4b9053 service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:03.346485 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:03.347441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:03.347726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:03.348002 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] PATH_INFO: `/v3/users/4fa2293dab3a462bbec42df70b5058e0` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:03.349201 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fa2293dab3a462bbec42df70b5058e0)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:03.351131 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:03.356486 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6fafb-2d25-4aee-9c32-d431ed4b9053 service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:03.361338 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:03.362287 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:03.362635 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:03.362972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] PATH_INFO: `/v3/users/e8058cfa8aa444208892078ae17abe0f` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:03.364187 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8058cfa8aa444208892078ae17abe0f)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:03.366083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:03.402091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fa2293dab3a462bbec42df70b5058e0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.409173 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fa2293dab3a462bbec42df70b5058e0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.409840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fa2293dab3a462bbec42df70b5058e0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.412258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8058cfa8aa444208892078ae17abe0f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.413035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8058cfa8aa444208892078ae17abe0f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.414006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8058cfa8aa444208892078ae17abe0f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.417643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fa2293dab3a462bbec42df70b5058e0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.418362 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4616f45c-fc59-4f86-8d63-2914589f2541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fa2293dab3a462bbec42df70b5058e0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.421811 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8058cfa8aa444208892078ae17abe0f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.428806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-11219c25-a572-4309-9e34-087e8e0532a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8058cfa8aa444208892078ae17abe0f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:03.437014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1354/2701] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:03 2022] DELETE /identity/v3/users/4fa2293dab3a462bbec42df70b5058e0 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:03.441651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28932f12-3726-49b3-a768-af5213727295 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:03.441939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28932f12-3726-49b3-a768-af5213727295 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:03.442223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28932f12-3726-49b3-a768-af5213727295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:03.446132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1348/2702] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:03 2022] DELETE /identity/v3/users/e8058cfa8aa444208892078ae17abe0f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:03.450184 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc03ca-861c-4fd5-8ae9-305116841506 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:03.450553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc03ca-861c-4fd5-8ae9-305116841506 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:03.450833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cc03ca-861c-4fd5-8ae9-305116841506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:03.459895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-28932f12-3726-49b3-a768-af5213727295 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:03.461099 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-28932f12-3726-49b3-a768-af5213727295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:03.468100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-55cc03ca-861c-4fd5-8ae9-305116841506 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:03.469262 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-55cc03ca-861c-4fd5-8ae9-305116841506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:03.481560 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:03.485404 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:03.523771 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1355/2703] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:03.530251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1349/2704] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:21:03.533512 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28932f12-3726-49b3-a768-af5213727295 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:03.534916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-28932f12-3726-49b3-a768-af5213727295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:03.539465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc03ca-861c-4fd5-8ae9-305116841506 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:03.541734 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-55cc03ca-861c-4fd5-8ae9-305116841506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:03.590423 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28932f12-3726-49b3-a768-af5213727295 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:03.594062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:03.594705 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:03.594983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:03.595251 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:03.597902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cc03ca-861c-4fd5-8ae9-305116841506 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:03.601670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:03.602322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:03.602609 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:03.602890 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:03.604562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:03.606023 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:03.612128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:03.613554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:03.615358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1356/2705] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:03.623170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1350/2706] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:03.893615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:03.895121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:03.909689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b7cc4-10cd-403c-9a74-654b78d40504 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:03.914685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:03.916687 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:03.930581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:03.931509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:03.931792 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:03.932067 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] PATH_INFO: `/v3/projects/bab48beabb5741edae35335d9fc8a4a9` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:03.934340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab48beabb5741edae35335d9fc8a4a9)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:03.936595 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:03.939349 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cfab-555e-4bbd-b4ea-137931f5a7bf service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:03.954852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:03.955779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:03.956065 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:03.956339 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] PATH_INFO: `/v3/projects/7977fb300d884296be5a96d690bfb610` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:03.958253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7977fb300d884296be5a96d690bfb610)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:03.960433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:04.000622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab48beabb5741edae35335d9fc8a4a9', 'request_id': 'req-3f18395b-9ecc-4599-a7b5-fd47087c04ef'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.007137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab48beabb5741edae35335d9fc8a4a9', 'request_id': 'req-3f18395b-9ecc-4599-a7b5-fd47087c04ef'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.015333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab48beabb5741edae35335d9fc8a4a9', 'request_id': 'req-3f18395b-9ecc-4599-a7b5-fd47087c04ef'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.018282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7977fb300d884296be5a96d690bfb610', 'request_id': 'req-9162f9eb-b709-4c0e-9208-b0a3628b0684'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.030643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7977fb300d884296be5a96d690bfb610', 'request_id': 'req-9162f9eb-b709-4c0e-9208-b0a3628b0684'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.033467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] The token cache is being invalidate because project bab48beabb5741edae35335d9fc8a4a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:04.033742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.034432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.043130 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1357/2707] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:03 2022] DELETE /identity/v3/projects/bab48beabb5741edae35335d9fc8a4a9 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:04.043433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7977fb300d884296be5a96d690bfb610', 'request_id': 'req-9162f9eb-b709-4c0e-9208-b0a3628b0684'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.049809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] The token cache is being invalidate because project 7977fb300d884296be5a96d690bfb610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:04.050202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.050939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.053038 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1351/2708] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:03 2022] DELETE /identity/v3/projects/7977fb300d884296be5a96d690bfb610 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:04.057723 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:04.059218 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:04.124348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:04.145211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f18395b-9ecc-4599-a7b5-fd47087c04ef admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:04.206547 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:04.207037 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:04.207425 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:04.207796 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] PATH_INFO: `/v3/users/de6a2580af1c4eb99f841adcc83496ce` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:04.209370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] RBAC: Authorizing `identity:delete_user(user_id=de6a2580af1c4eb99f841adcc83496ce)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:04.211740 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:04.270966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de6a2580af1c4eb99f841adcc83496ce'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.282568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de6a2580af1c4eb99f841adcc83496ce'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.283480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de6a2580af1c4eb99f841adcc83496ce'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.294745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de6a2580af1c4eb99f841adcc83496ce'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.296309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-13bddcfc-fd99-4681-a4c1-34d02c4eb18b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de6a2580af1c4eb99f841adcc83496ce'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.320332 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1358/2709] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:04 2022] DELETE /identity/v3/users/de6a2580af1c4eb99f841adcc83496ce => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:04.603798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:04.605965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:04.622082 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162f9eb-b709-4c0e-9208-b0a3628b0684 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:04.670122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:04.684939 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:04.685828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:04.686116 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:04.686421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] PATH_INFO: `/v3/projects/505c18e377e94b7db16b77c568d53e99` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:04.687843 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=505c18e377e94b7db16b77c568d53e99)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:04.690566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:04.738775 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505c18e377e94b7db16b77c568d53e99', 'request_id': 'req-4435325d-6450-4915-9386-0253efe68674'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.748925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505c18e377e94b7db16b77c568d53e99', 'request_id': 'req-4435325d-6450-4915-9386-0253efe68674'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.757543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505c18e377e94b7db16b77c568d53e99', 'request_id': 'req-4435325d-6450-4915-9386-0253efe68674'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.770476 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] The token cache is being invalidate because project 505c18e377e94b7db16b77c568d53e99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:04.770841 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.771862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4435325d-6450-4915-9386-0253efe68674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:04.776937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1352/2710] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:04 2022] DELETE /identity/v3/projects/505c18e377e94b7db16b77c568d53e99 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:04.778333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf87f09-78aa-4148-ae21-214ba95b6fc6 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:04.778667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf87f09-78aa-4148-ae21-214ba95b6fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:04.778950 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf87f09-78aa-4148-ae21-214ba95b6fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:04.797877 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-caf87f09-78aa-4148-ae21-214ba95b6fc6 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:04.799084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-caf87f09-78aa-4148-ae21-214ba95b6fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:04.860024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1359/2711] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 20 09:21:04.868284 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd13dd-1be8-4914-bc19-042e0a7894e2 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:04.868657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd13dd-1be8-4914-bc19-042e0a7894e2 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:04.869000 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bd13dd-1be8-4914-bc19-042e0a7894e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:04.888366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-15bd13dd-1be8-4914-bc19-042e0a7894e2 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:04.889683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-15bd13dd-1be8-4914-bc19-042e0a7894e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:04.898524 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1353/2712] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:04.902952 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf87f09-78aa-4148-ae21-214ba95b6fc6 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:04.912835 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf87f09-78aa-4148-ae21-214ba95b6fc6 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:04.917370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:04.918026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:04.918307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:04.918580 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:04.919350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:21:04.919658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:04.921274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:04.934021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1360/2713] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 20 09:21:04.939334 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd13dd-1be8-4914-bc19-042e0a7894e2 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:04.948998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bd13dd-1be8-4914-bc19-042e0a7894e2 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:04.955016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2c83d-7fd1-4bbf-899b-30866fe8440d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:04.955724 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2c83d-7fd1-4bbf-899b-30866fe8440d admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:04.955985 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2c83d-7fd1-4bbf-899b-30866fe8440d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:04.956258 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2c83d-7fd1-4bbf-899b-30866fe8440d admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:04.956960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2c83d-7fd1-4bbf-899b-30866fe8440d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:04.958923 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2c83d-7fd1-4bbf-899b-30866fe8440d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:04.992481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1354/2714] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:04 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:04.998797 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.018043 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894b173-b6e9-4886-ba15-d37f3061ac9a admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.022608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:05.083536 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32268198-19e0-49cc-85de-3e9fd4479e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.084206 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268198-19e0-49cc-85de-3e9fd4479e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.084490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268198-19e0-49cc-85de-3e9fd4479e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.084761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32268198-19e0-49cc-85de-3e9fd4479e5c admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.085428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32268198-19e0-49cc-85de-3e9fd4479e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.087468 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32268198-19e0-49cc-85de-3e9fd4479e5c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.109309 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1361/2715] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:04 2022] POST /identity/v3/users => generated 282 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:05.114821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2c83d-7fd1-4bbf-899b-30866fe8440d admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.125280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2c83d-7fd1-4bbf-899b-30866fe8440d admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.130288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59d22d-d6f0-4ec0-979a-3efa8a7c1857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.130950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59d22d-d6f0-4ec0-979a-3efa8a7c1857 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.131236 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59d22d-d6f0-4ec0-979a-3efa8a7c1857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.131559 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b59d22d-d6f0-4ec0-979a-3efa8a7c1857 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.132295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59d22d-d6f0-4ec0-979a-3efa8a7c1857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.133947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b59d22d-d6f0-4ec0-979a-3efa8a7c1857 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.145692 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1355/2716] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:05.149934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32268198-19e0-49cc-85de-3e9fd4479e5c admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.164939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32268198-19e0-49cc-85de-3e9fd4479e5c admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.171193 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e7785f-a328-4e0b-9a15-88a3070f46d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.171919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7785f-a328-4e0b-9a15-88a3070f46d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.172205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7785f-a328-4e0b-9a15-88a3070f46d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.172494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7785f-a328-4e0b-9a15-88a3070f46d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.173289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7785f-a328-4e0b-9a15-88a3070f46d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.176745 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e7785f-a328-4e0b-9a15-88a3070f46d7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.185864 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1362/2717] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:05.190102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59d22d-d6f0-4ec0-979a-3efa8a7c1857 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.200738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b59d22d-d6f0-4ec0-979a-3efa8a7c1857 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.206719 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75515a4e-47ec-4786-912f-6669e8ff3386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.207643 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75515a4e-47ec-4786-912f-6669e8ff3386 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.207925 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75515a4e-47ec-4786-912f-6669e8ff3386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.208204 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75515a4e-47ec-4786-912f-6669e8ff3386 admin admin] PATH_INFO: `/v3/projects/eb597029120a4fc1a4527ed117791b7a/users/baf8b3cb7ea74a169d0a57cd4992f571/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.223454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75515a4e-47ec-4786-912f-6669e8ff3386 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb597029120a4fc1a4527ed117791b7a, user_id=baf8b3cb7ea74a169d0a57cd4992f571, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.226112 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75515a4e-47ec-4786-912f-6669e8ff3386 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.239835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1356/2718] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:05 2022] PUT /identity/v3/projects/eb597029120a4fc1a4527ed117791b7a/users/baf8b3cb7ea74a169d0a57cd4992f571/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:05.251545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441207a-13ca-4e14-8274-fed888ec3a48 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.251842 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441207a-13ca-4e14-8274-fed888ec3a48 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.252120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441207a-13ca-4e14-8274-fed888ec3a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.282166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5441207a-13ca-4e14-8274-fed888ec3a48 None None] MFA Rules not processed for user `baf8b3cb7ea74a169d0a57cd4992f571`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:05.284169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5441207a-13ca-4e14-8274-fed888ec3a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:05.359958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1363/2719] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:05 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:05.364627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75515a4e-47ec-4786-912f-6669e8ff3386 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.446250 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75515a4e-47ec-4786-912f-6669e8ff3386 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.451076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bc61e-18d5-4b74-ba23-ff9cc1927399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.451782 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bc61e-18d5-4b74-ba23-ff9cc1927399 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.452066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bc61e-18d5-4b74-ba23-ff9cc1927399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.452396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9bc61e-18d5-4b74-ba23-ff9cc1927399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.453081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bc61e-18d5-4b74-ba23-ff9cc1927399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.455005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9bc61e-18d5-4b74-ba23-ff9cc1927399 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.476747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:05.484889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1357/2720] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:05 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:21:05.489046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441207a-13ca-4e14-8274-fed888ec3a48 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.501228 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5441207a-13ca-4e14-8274-fed888ec3a48 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.556077 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e35f5-82d5-4a25-8ae2-27e22a23afc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.556736 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e35f5-82d5-4a25-8ae2-27e22a23afc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.557018 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e35f5-82d5-4a25-8ae2-27e22a23afc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.557291 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e35f5-82d5-4a25-8ae2-27e22a23afc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.557996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e35f5-82d5-4a25-8ae2-27e22a23afc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.559983 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e35f5-82d5-4a25-8ae2-27e22a23afc6 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.580157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1364/2721] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:05 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:21:05.584346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bc61e-18d5-4b74-ba23-ff9cc1927399 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.596061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9bc61e-18d5-4b74-ba23-ff9cc1927399 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.601196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78f382-9fc6-4e12-bf18-8e5daf8dc377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.601864 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78f382-9fc6-4e12-bf18-8e5daf8dc377 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.602147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78f382-9fc6-4e12-bf18-8e5daf8dc377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.602416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed78f382-9fc6-4e12-bf18-8e5daf8dc377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.603147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78f382-9fc6-4e12-bf18-8e5daf8dc377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.604802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed78f382-9fc6-4e12-bf18-8e5daf8dc377 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.616994 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1358/2722] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:05.621342 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e35f5-82d5-4a25-8ae2-27e22a23afc6 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.627981 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:05.631989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e35f5-82d5-4a25-8ae2-27e22a23afc6 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.638443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f40ef-13b7-4341-9c1c-70f29d0722f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.639478 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f40ef-13b7-4341-9c1c-70f29d0722f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.639802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f40ef-13b7-4341-9c1c-70f29d0722f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.640209 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f40ef-13b7-4341-9c1c-70f29d0722f7 admin admin] PATH_INFO: `/v3/projects/ea78d26fce194f78abed62cec9877572/users/b6f83873e2e2497baa36a626a0c9806d/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.655928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f40ef-13b7-4341-9c1c-70f29d0722f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea78d26fce194f78abed62cec9877572, user_id=b6f83873e2e2497baa36a626a0c9806d, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.658589 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f40ef-13b7-4341-9c1c-70f29d0722f7 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.673657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1365/2723] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:05 2022] PUT /identity/v3/projects/ea78d26fce194f78abed62cec9877572/users/b6f83873e2e2497baa36a626a0c9806d/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:05.677750 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78f382-9fc6-4e12-bf18-8e5daf8dc377 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.687889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed78f382-9fc6-4e12-bf18-8e5daf8dc377 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.744045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a828-a001-409f-998f-9aa99ef039fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.744732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a828-a001-409f-998f-9aa99ef039fe admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.745017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a828-a001-409f-998f-9aa99ef039fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.745288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a828-a001-409f-998f-9aa99ef039fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.746066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a828-a001-409f-998f-9aa99ef039fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.747720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a828-a001-409f-998f-9aa99ef039fe admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.757838 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1359/2724] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:05.762296 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f40ef-13b7-4341-9c1c-70f29d0722f7 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.773754 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f40ef-13b7-4341-9c1c-70f29d0722f7 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.778664 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.779622 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.779903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.780211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] PATH_INFO: `/v3/projects/ea78d26fce194f78abed62cec9877572/users/b6f83873e2e2497baa36a626a0c9806d/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.782230 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea78d26fce194f78abed62cec9877572, user_id=b6f83873e2e2497baa36a626a0c9806d, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.784921 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:05.796129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1366/2725] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:05 2022] PUT /identity/v3/projects/ea78d26fce194f78abed62cec9877572/users/b6f83873e2e2497baa36a626a0c9806d/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:05.806531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120da3f9-e4cf-4a85-9b89-55aaac76cf3b None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.806794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120da3f9-e4cf-4a85-9b89-55aaac76cf3b None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.807086 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120da3f9-e4cf-4a85-9b89-55aaac76cf3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.832896 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-120da3f9-e4cf-4a85-9b89-55aaac76cf3b None None] MFA Rules not processed for user `b6f83873e2e2497baa36a626a0c9806d`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:05.834100 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-120da3f9-e4cf-4a85-9b89-55aaac76cf3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:05.902213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1360/2726] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:05 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:05.914660 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:05.916804 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:05.976667 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b052cd2-f2e6-42d4-be83-3b90bdb2d4b4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:05.980557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533160-b3e2-4037-9bb4-8df064a502c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:05.981186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533160-b3e2-4037-9bb4-8df064a502c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:05.981457 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533160-b3e2-4037-9bb4-8df064a502c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:05.981737 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9533160-b3e2-4037-9bb4-8df064a502c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:05.992571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9533160-b3e2-4037-9bb4-8df064a502c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:05.994747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9533160-b3e2-4037-9bb4-8df064a502c0 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:06.006136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1367/2727] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:05 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:06.092232 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120da3f9-e4cf-4a85-9b89-55aaac76cf3b None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:06.102333 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120da3f9-e4cf-4a85-9b89-55aaac76cf3b None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:06.106388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04496f56-f8f5-4b18-994f-300cde4f25d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:06.107035 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04496f56-f8f5-4b18-994f-300cde4f25d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:06.107313 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04496f56-f8f5-4b18-994f-300cde4f25d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:06.107616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04496f56-f8f5-4b18-994f-300cde4f25d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:06.167408 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04496f56-f8f5-4b18-994f-300cde4f25d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:06.169439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04496f56-f8f5-4b18-994f-300cde4f25d0 service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:06.184295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1361/2728] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:06 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:11.989789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533160-b3e2-4037-9bb4-8df064a502c0 service cinder] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:12.050365 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9533160-b3e2-4037-9bb4-8df064a502c0 service cinder] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:12.055221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:12.056112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:12.056394 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:12.056665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] PATH_INFO: `/v3/users/baf8b3cb7ea74a169d0a57cd4992f571` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:12.057855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baf8b3cb7ea74a169d0a57cd4992f571)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:12.059838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:12.109071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baf8b3cb7ea74a169d0a57cd4992f571'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:12.116432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baf8b3cb7ea74a169d0a57cd4992f571'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:12.117044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baf8b3cb7ea74a169d0a57cd4992f571'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:12.128290 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baf8b3cb7ea74a169d0a57cd4992f571'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:12.128959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baf8b3cb7ea74a169d0a57cd4992f571'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:12.136432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:12.149793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1368/2729] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:11 2022] DELETE /identity/v3/users/baf8b3cb7ea74a169d0a57cd4992f571 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:12.154475 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7780db4-2c2c-4d3d-8ec7-f3178b1af432 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:12.154940 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7780db4-2c2c-4d3d-8ec7-f3178b1af432 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:12.155176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7780db4-2c2c-4d3d-8ec7-f3178b1af432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:12.173462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-d7780db4-2c2c-4d3d-8ec7-f3178b1af432 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:12.174944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d7780db4-2c2c-4d3d-8ec7-f3178b1af432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:12.179272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:12.236573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1362/2730] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:12.246552 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:12.248075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:12.306991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc04a19-b4c3-47c6-808b-0c1c208e5005 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:12.310926 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a6a21-113b-498a-b7e4-ef3d1b26e587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:12.311604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a6a21-113b-498a-b7e4-ef3d1b26e587 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:12.311882 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a6a21-113b-498a-b7e4-ef3d1b26e587 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:12.312157 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a6a21-113b-498a-b7e4-ef3d1b26e587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:12.321674 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a6a21-113b-498a-b7e4-ef3d1b26e587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:12.323135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a6a21-113b-498a-b7e4-ef3d1b26e587 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:12.335011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1369/2731] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:12.900201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7780db4-2c2c-4d3d-8ec7-f3178b1af432 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:12.901667 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d7780db4-2c2c-4d3d-8ec7-f3178b1af432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:12.916145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7780db4-2c2c-4d3d-8ec7-f3178b1af432 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:12.921196 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:12.922071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:12.922362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:12.922670 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] PATH_INFO: `/v3/projects/eb597029120a4fc1a4527ed117791b7a` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:12.924027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb597029120a4fc1a4527ed117791b7a)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:12.926081 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:12.974492 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb597029120a4fc1a4527ed117791b7a', 'request_id': 'req-0364b344-6230-4bea-be9b-39ceefd60ef8'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:12.984113 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb597029120a4fc1a4527ed117791b7a', 'request_id': 'req-0364b344-6230-4bea-be9b-39ceefd60ef8'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:12.990441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb597029120a4fc1a4527ed117791b7a', 'request_id': 'req-0364b344-6230-4bea-be9b-39ceefd60ef8'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.003107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] The token cache is being invalidate because project eb597029120a4fc1a4527ed117791b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:13.003466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.004189 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.005816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1363/2732] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:12 2022] DELETE /identity/v3/projects/eb597029120a4fc1a4527ed117791b7a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:13.010009 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a6a21-113b-498a-b7e4-ef3d1b26e587 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:13.020429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a6a21-113b-498a-b7e4-ef3d1b26e587 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:13.087931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:13.088812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.089096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.089374 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] PATH_INFO: `/v3/users/b6f83873e2e2497baa36a626a0c9806d` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.090599 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6f83873e2e2497baa36a626a0c9806d)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:13.092571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:13.142175 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6f83873e2e2497baa36a626a0c9806d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.149053 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6f83873e2e2497baa36a626a0c9806d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.149696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6f83873e2e2497baa36a626a0c9806d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.158786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6f83873e2e2497baa36a626a0c9806d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.159487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a9d862a1-42a5-4ff6-bd2b-3b382051fc78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6f83873e2e2497baa36a626a0c9806d'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.177339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1370/2733] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:13 2022] DELETE /identity/v3/users/b6f83873e2e2497baa36a626a0c9806d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:13.449073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:13.450638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:13.465898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0364b344-6230-4bea-be9b-39ceefd60ef8 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:13.528580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:13.529473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.529753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.530030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] PATH_INFO: `/v3/projects/ea78d26fce194f78abed62cec9877572` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.531371 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea78d26fce194f78abed62cec9877572)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:13.533411 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:13.573172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:13.586644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea78d26fce194f78abed62cec9877572', 'request_id': 'req-5c9d8744-caa4-48b3-b732-b78bc517a1ed'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.597102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea78d26fce194f78abed62cec9877572', 'request_id': 'req-5c9d8744-caa4-48b3-b732-b78bc517a1ed'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.602686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea78d26fce194f78abed62cec9877572', 'request_id': 'req-5c9d8744-caa4-48b3-b732-b78bc517a1ed'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.613580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] The token cache is being invalidate because project ea78d26fce194f78abed62cec9877572 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:13.613877 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.614497 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5c9d8744-caa4-48b3-b732-b78bc517a1ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:13.616217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1364/2734] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:13 2022] DELETE /identity/v3/projects/ea78d26fce194f78abed62cec9877572 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:13.621318 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa43f-383d-4a43-a767-03afba15e6df None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.621606 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa43f-383d-4a43-a767-03afba15e6df None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.621883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fa43f-383d-4a43-a767-03afba15e6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.641268 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-cf9fa43f-383d-4a43-a767-03afba15e6df None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:13.642591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cf9fa43f-383d-4a43-a767-03afba15e6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:13.648256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:13.701327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1371/2735] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:13.710348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f9803-c348-43fc-90b2-5cf7cf59d258 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.710634 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f9803-c348-43fc-90b2-5cf7cf59d258 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.710907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f9803-c348-43fc-90b2-5cf7cf59d258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.730466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-d41f9803-c348-43fc-90b2-5cf7cf59d258 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:13.731713 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d41f9803-c348-43fc-90b2-5cf7cf59d258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:13.740542 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1365/2736] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:13.745090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa43f-383d-4a43-a767-03afba15e6df None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:13.754494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fa43f-383d-4a43-a767-03afba15e6df None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:13.759407 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:13.760063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.760353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.760597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.761322 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80638) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:21:13.761627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:13.763205 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:13.780990 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1372/2737] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:21:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:21:13.785589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f9803-c348-43fc-90b2-5cf7cf59d258 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:13.795006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f9803-c348-43fc-90b2-5cf7cf59d258 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:13.800344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7ec4e-b6b4-4e4d-9cb2-9faaba187f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:13.800996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7ec4e-b6b4-4e4d-9cb2-9faaba187f16 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.801277 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7ec4e-b6b4-4e4d-9cb2-9faaba187f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.801557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a7ec4e-b6b4-4e4d-9cb2-9faaba187f16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.802244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7ec4e-b6b4-4e4d-9cb2-9faaba187f16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:13.804223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a7ec4e-b6b4-4e4d-9cb2-9faaba187f16 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:13.831610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1366/2738] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:13 2022] POST /identity/v3/projects => generated 387 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:13.836051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:13.846182 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbc213b-2324-4fdb-aede-d28f7f919b4b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:13.901604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38898fa-cad4-4cb8-8112-0c1cb4b9533f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:13.902273 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38898fa-cad4-4cb8-8112-0c1cb4b9533f admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.902549 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38898fa-cad4-4cb8-8112-0c1cb4b9533f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.902827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38898fa-cad4-4cb8-8112-0c1cb4b9533f admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.903540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38898fa-cad4-4cb8-8112-0c1cb4b9533f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:13.905460 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38898fa-cad4-4cb8-8112-0c1cb4b9533f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:13.928078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1373/2739] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:13 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:13.932227 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7ec4e-b6b4-4e4d-9cb2-9faaba187f16 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:13.942660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a7ec4e-b6b4-4e4d-9cb2-9faaba187f16 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:13.947499 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f170da8-003b-4def-8046-12f00c05b954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:13.948160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f170da8-003b-4def-8046-12f00c05b954 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.948439 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f170da8-003b-4def-8046-12f00c05b954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.948715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f170da8-003b-4def-8046-12f00c05b954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.949456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f170da8-003b-4def-8046-12f00c05b954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:13.951066 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f170da8-003b-4def-8046-12f00c05b954 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:13.961469 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1367/2740] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:13.965141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38898fa-cad4-4cb8-8112-0c1cb4b9533f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:13.974988 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38898fa-cad4-4cb8-8112-0c1cb4b9533f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:13.979942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b016c-dd2e-4eb5-9351-c82cdfc9b0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:13.980634 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b016c-dd2e-4eb5-9351-c82cdfc9b0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:13.980918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b016c-dd2e-4eb5-9351-c82cdfc9b0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:13.981194 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b016c-dd2e-4eb5-9351-c82cdfc9b0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:13.981949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b016c-dd2e-4eb5-9351-c82cdfc9b0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:13.983608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097b016c-dd2e-4eb5-9351-c82cdfc9b0e2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:13.995825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1374/2741] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:14.000191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f170da8-003b-4def-8046-12f00c05b954 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:14.012953 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f170da8-003b-4def-8046-12f00c05b954 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:14.018975 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ee4ff-8425-43dc-9fc6-9f5fe8dc652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:14.020042 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee4ff-8425-43dc-9fc6-9f5fe8dc652e admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.020344 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee4ff-8425-43dc-9fc6-9f5fe8dc652e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.020619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ee4ff-8425-43dc-9fc6-9f5fe8dc652e admin admin] PATH_INFO: `/v3/projects/d3a4b2d31e63423695212df4f97cb683/users/661ff549fbe84715ad779af9a97c0c41/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.039731 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ee4ff-8425-43dc-9fc6-9f5fe8dc652e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3a4b2d31e63423695212df4f97cb683, user_id=661ff549fbe84715ad779af9a97c0c41, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:14.042406 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8ee4ff-8425-43dc-9fc6-9f5fe8dc652e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:14.057445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1368/2742] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:13 2022] PUT /identity/v3/projects/d3a4b2d31e63423695212df4f97cb683/users/661ff549fbe84715ad779af9a97c0c41/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:14.068701 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd47e-92d6-47c8-b94b-9d1579a76f50 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.069190 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd47e-92d6-47c8-b94b-9d1579a76f50 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.069306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd47e-92d6-47c8-b94b-9d1579a76f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.096531 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-057dd47e-92d6-47c8-b94b-9d1579a76f50 None None] MFA Rules not processed for user `661ff549fbe84715ad779af9a97c0c41`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:14.097735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-057dd47e-92d6-47c8-b94b-9d1579a76f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:14.161693 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1375/2743] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:14.166493 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ee4ff-8425-43dc-9fc6-9f5fe8dc652e admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:14.230952 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ee4ff-8425-43dc-9fc6-9f5fe8dc652e admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:14.236150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de43802-4445-4c72-93ba-53c081eb5915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:14.236817 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43802-4445-4c72-93ba-53c081eb5915 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.237098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43802-4445-4c72-93ba-53c081eb5915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.237388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de43802-4445-4c72-93ba-53c081eb5915 admin admin] PATH_INFO: `/v3/projects` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.238061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de43802-4445-4c72-93ba-53c081eb5915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:14.240003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de43802-4445-4c72-93ba-53c081eb5915 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:14.268848 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1369/2744] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:14 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:14.273274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dd47e-92d6-47c8-b94b-9d1579a76f50 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:14.284247 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057dd47e-92d6-47c8-b94b-9d1579a76f50 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:14.288008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:14.346024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1fcc-9558-4a38-84e0-702a763058a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:14.346850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1fcc-9558-4a38-84e0-702a763058a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.347134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1fcc-9558-4a38-84e0-702a763058a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.347443 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e1fcc-9558-4a38-84e0-702a763058a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.348146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e1fcc-9558-4a38-84e0-702a763058a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:14.350112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e1fcc-9558-4a38-84e0-702a763058a9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:14.373432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1376/2745] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:14 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:14.377755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de43802-4445-4c72-93ba-53c081eb5915 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:14.388509 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de43802-4445-4c72-93ba-53c081eb5915 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:14.394365 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704876c3-317b-4391-9ef1-d96ab3514f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:14.395025 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704876c3-317b-4391-9ef1-d96ab3514f8b admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.395337 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704876c3-317b-4391-9ef1-d96ab3514f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.395613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704876c3-317b-4391-9ef1-d96ab3514f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.396378 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704876c3-317b-4391-9ef1-d96ab3514f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:14.398017 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704876c3-317b-4391-9ef1-d96ab3514f8b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:14.408534 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1370/2746] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:14.412697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1fcc-9558-4a38-84e0-702a763058a9 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:14.423145 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e1fcc-9558-4a38-84e0-702a763058a9 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:14.428712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25179d99-b5e2-4ba0-9a98-eb89c1c61dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:14.429592 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25179d99-b5e2-4ba0-9a98-eb89c1c61dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.429910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25179d99-b5e2-4ba0-9a98-eb89c1c61dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.430143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25179d99-b5e2-4ba0-9a98-eb89c1c61dd4 admin admin] PATH_INFO: `/v3/projects/b15eded0f5ba4d81b97e9bddeb9a8190/users/b2cbc9c6e55b4382a7a38666e5e56aef/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.446863 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25179d99-b5e2-4ba0-9a98-eb89c1c61dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15eded0f5ba4d81b97e9bddeb9a8190, user_id=b2cbc9c6e55b4382a7a38666e5e56aef, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:14.449626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25179d99-b5e2-4ba0-9a98-eb89c1c61dd4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:14.465502 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1377/2747] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:14 2022] PUT /identity/v3/projects/b15eded0f5ba4d81b97e9bddeb9a8190/users/b2cbc9c6e55b4382a7a38666e5e56aef/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:14.469647 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704876c3-317b-4391-9ef1-d96ab3514f8b admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:14.480662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704876c3-317b-4391-9ef1-d96ab3514f8b admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:14.512173 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:14.542385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9324a4-d7d3-4eda-941f-cdd1a6584da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:14.543036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9324a4-d7d3-4eda-941f-cdd1a6584da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.543354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9324a4-d7d3-4eda-941f-cdd1a6584da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.543628 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9324a4-d7d3-4eda-941f-cdd1a6584da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.544361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9324a4-d7d3-4eda-941f-cdd1a6584da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:14.545966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9324a4-d7d3-4eda-941f-cdd1a6584da8 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:14.555930 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1371/2748] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:14.560249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25179d99-b5e2-4ba0-9a98-eb89c1c61dd4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:14.570782 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25179d99-b5e2-4ba0-9a98-eb89c1c61dd4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:14.575724 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:14.576613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.576889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.577160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] PATH_INFO: `/v3/projects/b15eded0f5ba4d81b97e9bddeb9a8190/users/b2cbc9c6e55b4382a7a38666e5e56aef/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.579249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b15eded0f5ba4d81b97e9bddeb9a8190, user_id=b2cbc9c6e55b4382a7a38666e5e56aef, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:14.581971 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:14.592355 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1378/2749] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:14 2022] PUT /identity/v3/projects/b15eded0f5ba4d81b97e9bddeb9a8190/users/b2cbc9c6e55b4382a7a38666e5e56aef/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:14.602630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb877d2-931d-4404-9391-ef9e4a8488ed None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.602920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb877d2-931d-4404-9391-ef9e4a8488ed None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.603195 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb877d2-931d-4404-9391-ef9e4a8488ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.632581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-0cb877d2-931d-4404-9391-ef9e4a8488ed None None] MFA Rules not processed for user `b2cbc9c6e55b4382a7a38666e5e56aef`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:14.633863 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0cb877d2-931d-4404-9391-ef9e4a8488ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:14.705568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1372/2750] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:14.716436 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:14.718402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:14.779712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e4cb55-fa7d-40ca-8710-3c7770dad532 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:14.784495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab7875-dbf6-4291-9b6a-b5a4ad69b10a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:14.785139 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7875-dbf6-4291-9b6a-b5a4ad69b10a service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:14.785427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7875-dbf6-4291-9b6a-b5a4ad69b10a service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:14.785700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab7875-dbf6-4291-9b6a-b5a4ad69b10a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:14.796521 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab7875-dbf6-4291-9b6a-b5a4ad69b10a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:14.797974 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab7875-dbf6-4291-9b6a-b5a4ad69b10a service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:14.809326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1379/2751] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:15.398821 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb877d2-931d-4404-9391-ef9e4a8488ed None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:15.462745 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb877d2-931d-4404-9391-ef9e4a8488ed None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:15.467956 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:15.468849 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:15.469127 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:15.469401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] PATH_INFO: `/v3/users/661ff549fbe84715ad779af9a97c0c41` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:15.470683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] RBAC: Authorizing `identity:delete_user(user_id=661ff549fbe84715ad779af9a97c0c41)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:15.472678 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:15.525998 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661ff549fbe84715ad779af9a97c0c41'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:15.526664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661ff549fbe84715ad779af9a97c0c41'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:15.527296 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661ff549fbe84715ad779af9a97c0c41'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:15.537413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661ff549fbe84715ad779af9a97c0c41'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:15.540119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:15.547361 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661ff549fbe84715ad779af9a97c0c41'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:15.565676 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1373/2752] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:15 2022] DELETE /identity/v3/users/661ff549fbe84715ad779af9a97c0c41 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:15.570534 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb50e91-2b6a-4764-989a-b81b3d7e9e79 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:15.571045 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb50e91-2b6a-4764-989a-b81b3d7e9e79 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:15.571144 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb50e91-2b6a-4764-989a-b81b3d7e9e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:15.589352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2eb50e91-2b6a-4764-989a-b81b3d7e9e79 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:15.590539 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2eb50e91-2b6a-4764-989a-b81b3d7e9e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:15.651637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1380/2753] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:15.663089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:15.664789 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:15.721788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb23633-53e9-43d7-918a-f7f8ff3d698d admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:15.726083 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53072b6-796b-49d6-8076-0806ccf09ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:15.726725 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53072b6-796b-49d6-8076-0806ccf09ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:15.727004 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53072b6-796b-49d6-8076-0806ccf09ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:15.727289 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53072b6-796b-49d6-8076-0806ccf09ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:15.736972 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53072b6-796b-49d6-8076-0806ccf09ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:15.739218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53072b6-796b-49d6-8076-0806ccf09ae0 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:15.749111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1374/2754] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:16.030978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb50e91-2b6a-4764-989a-b81b3d7e9e79 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:16.032466 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2eb50e91-2b6a-4764-989a-b81b3d7e9e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:16.042569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:16.050123 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb50e91-2b6a-4764-989a-b81b3d7e9e79 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:16.057353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:16.058370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:16.058976 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:16.059371 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] PATH_INFO: `/v3/projects/d3a4b2d31e63423695212df4f97cb683` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:16.061066 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3a4b2d31e63423695212df4f97cb683)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:16.064044 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:16.112151 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3a4b2d31e63423695212df4f97cb683', 'request_id': 'req-8f916488-f221-4cf2-8b56-2d534ab01af1'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.119088 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3a4b2d31e63423695212df4f97cb683', 'request_id': 'req-8f916488-f221-4cf2-8b56-2d534ab01af1'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.125353 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3a4b2d31e63423695212df4f97cb683', 'request_id': 'req-8f916488-f221-4cf2-8b56-2d534ab01af1'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.139274 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] The token cache is being invalidate because project d3a4b2d31e63423695212df4f97cb683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:16.139571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.140523 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.142078 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1381/2755] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:16 2022] DELETE /identity/v3/projects/d3a4b2d31e63423695212df4f97cb683 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:16.146383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53072b6-796b-49d6-8076-0806ccf09ae0 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:16.156614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53072b6-796b-49d6-8076-0806ccf09ae0 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:16.234403 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:16.235409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:16.235591 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:16.235865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] PATH_INFO: `/v3/users/b2cbc9c6e55b4382a7a38666e5e56aef` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:16.237370 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2cbc9c6e55b4382a7a38666e5e56aef)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:16.243016 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:16.295753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2cbc9c6e55b4382a7a38666e5e56aef'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.296533 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2cbc9c6e55b4382a7a38666e5e56aef'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.297191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2cbc9c6e55b4382a7a38666e5e56aef'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.306242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2cbc9c6e55b4382a7a38666e5e56aef'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.315318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-b20245da-2e9c-4785-98b6-87f2859123a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2cbc9c6e55b4382a7a38666e5e56aef'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.327231 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:16.336448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1375/2756] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:16 2022] DELETE /identity/v3/users/b2cbc9c6e55b4382a7a38666e5e56aef => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:16.631112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:16.632850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:16.652794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f916488-f221-4cf2-8b56-2d534ab01af1 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:16.709126 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:16.709986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:16.710261 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:16.710532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] PATH_INFO: `/v3/projects/b15eded0f5ba4d81b97e9bddeb9a8190` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:16.711733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b15eded0f5ba4d81b97e9bddeb9a8190)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:16.713643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:16.770826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b15eded0f5ba4d81b97e9bddeb9a8190', 'request_id': 'req-4a5f87ed-99c1-4297-a921-bf3fb86b8453'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.781992 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b15eded0f5ba4d81b97e9bddeb9a8190', 'request_id': 'req-4a5f87ed-99c1-4297-a921-bf3fb86b8453'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.788604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b15eded0f5ba4d81b97e9bddeb9a8190', 'request_id': 'req-4a5f87ed-99c1-4297-a921-bf3fb86b8453'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.803879 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] The token cache is being invalidate because project b15eded0f5ba4d81b97e9bddeb9a8190 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:16.804162 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.804783 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4a5f87ed-99c1-4297-a921-bf3fb86b8453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:16.806417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1382/2757] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:16 2022] DELETE /identity/v3/projects/b15eded0f5ba4d81b97e9bddeb9a8190 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:16.812512 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7ed6-4712-46fe-a1a2-9a860d4abd78 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:16.812990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7ed6-4712-46fe-a1a2-9a860d4abd78 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:16.813104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afa7ed6-4712-46fe-a1a2-9a860d4abd78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:16.832256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4afa7ed6-4712-46fe-a1a2-9a860d4abd78 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:16.833459 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4afa7ed6-4712-46fe-a1a2-9a860d4abd78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:16.896554 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1376/2758] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:16.904673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b65c0-a705-4414-8865-7183f9747154 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:16.904958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b65c0-a705-4414-8865-7183f9747154 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:16.905233 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b65c0-a705-4414-8865-7183f9747154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:16.924084 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-a69b65c0-a705-4414-8865-7183f9747154 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:16.925350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-a69b65c0-a705-4414-8865-7183f9747154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:16.933889 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1383/2759] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:16.938217 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa7ed6-4712-46fe-a1a2-9a860d4abd78 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:16.948665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afa7ed6-4712-46fe-a1a2-9a860d4abd78 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:16.953657 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:16.954309 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:16.954589 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:16.954862 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:16.955626 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:21:16.955932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:16.957761 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:16.969704 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1377/2760] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:21:16.973923 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b65c0-a705-4414-8865-7183f9747154 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:16.983483 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b65c0-a705-4414-8865-7183f9747154 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:16.988819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcfdbb-557a-4311-bb49-91f23885f9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:16.989455 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfdbb-557a-4311-bb49-91f23885f9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:16.989738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfdbb-557a-4311-bb49-91f23885f9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:16.990008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcfdbb-557a-4311-bb49-91f23885f9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:16.990695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcfdbb-557a-4311-bb49-91f23885f9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:16.992620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcfdbb-557a-4311-bb49-91f23885f9d3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:16.996837 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:17.022928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1384/2761] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:16 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:17.027270 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:17.037488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93bd25c-d057-4155-88a0-69ae4be886b0 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:17.107122 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b39850-03f1-4ad6-8b38-8c9d08bcdde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:17.107851 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b39850-03f1-4ad6-8b38-8c9d08bcdde7 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:17.108133 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b39850-03f1-4ad6-8b38-8c9d08bcdde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:17.108424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b39850-03f1-4ad6-8b38-8c9d08bcdde7 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:17.109142 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b39850-03f1-4ad6-8b38-8c9d08bcdde7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:17.111060 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b39850-03f1-4ad6-8b38-8c9d08bcdde7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:17.136095 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1378/2762] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:17 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:17.140759 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcfdbb-557a-4311-bb49-91f23885f9d3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:17.151638 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcfdbb-557a-4311-bb49-91f23885f9d3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:17.156593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660fdfd-6819-4abc-a464-222f1131fca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:17.157256 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660fdfd-6819-4abc-a464-222f1131fca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:17.157542 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660fdfd-6819-4abc-a464-222f1131fca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:17.157817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6660fdfd-6819-4abc-a464-222f1131fca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:17.158543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660fdfd-6819-4abc-a464-222f1131fca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:17.160642 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6660fdfd-6819-4abc-a464-222f1131fca0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:17.170657 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1385/2763] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:17.174816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b39850-03f1-4ad6-8b38-8c9d08bcdde7 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:17.185885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b39850-03f1-4ad6-8b38-8c9d08bcdde7 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:17.190876 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140bf1e-a518-4370-85e2-5aba3637f5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:17.191552 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140bf1e-a518-4370-85e2-5aba3637f5d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:17.191878 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140bf1e-a518-4370-85e2-5aba3637f5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:17.192132 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140bf1e-a518-4370-85e2-5aba3637f5d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:17.192900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140bf1e-a518-4370-85e2-5aba3637f5d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:17.194508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3140bf1e-a518-4370-85e2-5aba3637f5d9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:17.204563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1379/2764] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:17.208590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660fdfd-6819-4abc-a464-222f1131fca0 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:17.218618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6660fdfd-6819-4abc-a464-222f1131fca0 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:17.223735 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:17.224603 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:17.224884 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:17.225180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] PATH_INFO: `/v3/projects/5eeca6a310014d5c881a144f59a5f329/users/5e591d6211504fb4a2a54e260c2f117a/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:17.240259 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eeca6a310014d5c881a144f59a5f329, user_id=5e591d6211504fb4a2a54e260c2f117a, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:17.242901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:17.259929 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1386/2765] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:17 2022] PUT /identity/v3/projects/5eeca6a310014d5c881a144f59a5f329/users/5e591d6211504fb4a2a54e260c2f117a/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:17.271467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2d02-2fff-4547-9222-f1406616346f None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:17.271780 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2d02-2fff-4547-9222-f1406616346f None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:17.272059 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48a2d02-2fff-4547-9222-f1406616346f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:17.299301 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-d48a2d02-2fff-4547-9222-f1406616346f None None] MFA Rules not processed for user `5e591d6211504fb4a2a54e260c2f117a`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:17.300568 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d48a2d02-2fff-4547-9222-f1406616346f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:17.352551 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:17.370846 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1380/2766] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:17.382494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:17.384433 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:17.447413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf715d7-78e8-40d6-ab1d-2bccd2b1ecb0 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:17.453718 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17566f-7ef0-4be0-a860-05a7f4afea99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d25a45f9ee4ae58c6751a8781d12d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:17.454649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17566f-7ef0-4be0-a860-05a7f4afea99 service glance] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:17.455010 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17566f-7ef0-4be0-a860-05a7f4afea99 service glance] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:17.455307 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de17566f-7ef0-4be0-a860-05a7f4afea99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:17.464550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17566f-7ef0-4be0-a860-05a7f4afea99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:17.466186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de17566f-7ef0-4be0-a860-05a7f4afea99 service glance] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:17.475714 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1387/2767] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:17 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:18.631346 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a2d02-2fff-4547-9222-f1406616346f None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:18.633912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-d48a2d02-2fff-4547-9222-f1406616346f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:18.712272 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48a2d02-2fff-4547-9222-f1406616346f None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:18.716962 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:18.717856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:18.718134 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:18.718410 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] PATH_INFO: `/v3/users/27489c3dc91649eba176fb290f3e96a3` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:18.719764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=27489c3dc91649eba176fb290f3e96a3)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:18.721802 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:18.778357 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27489c3dc91649eba176fb290f3e96a3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:18.779191 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27489c3dc91649eba176fb290f3e96a3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:18.779912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27489c3dc91649eba176fb290f3e96a3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:18.789834 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27489c3dc91649eba176fb290f3e96a3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:18.796933 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27489c3dc91649eba176fb290f3e96a3'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:18.814986 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1381/2768] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:18 2022] DELETE /identity/v3/users/27489c3dc91649eba176fb290f3e96a3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:19.004361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17566f-7ef0-4be0-a860-05a7f4afea99 service glance] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:19.017487 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de17566f-7ef0-4be0-a860-05a7f4afea99 service glance] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:19.092127 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:19.092993 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:19.093272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:19.093545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] PATH_INFO: `/v3/projects/aac6a376528d47d5a00b91de9b2df9d0` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:19.094906 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac6a376528d47d5a00b91de9b2df9d0)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:19.096991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:19.162462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac6a376528d47d5a00b91de9b2df9d0', 'request_id': 'req-86440fd9-8967-45cd-871b-836b59c2e1c3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.167817 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:19.169333 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac6a376528d47d5a00b91de9b2df9d0', 'request_id': 'req-86440fd9-8967-45cd-871b-836b59c2e1c3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.176340 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac6a376528d47d5a00b91de9b2df9d0', 'request_id': 'req-86440fd9-8967-45cd-871b-836b59c2e1c3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.191326 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] The token cache is being invalidate because project aac6a376528d47d5a00b91de9b2df9d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:19.191618 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.192292 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.193910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1388/2769] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:19 2022] DELETE /identity/v3/projects/aac6a376528d47d5a00b91de9b2df9d0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:19.198009 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:19.199445 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:19.212617 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce843bd-6d0b-4751-9448-326a0a1922bc admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:19.275433 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:19.276323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:19.276598 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:19.276866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] PATH_INFO: `/v3/users/114ed6a689514c058f36ef08e3cb9f0b` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:19.278135 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=114ed6a689514c058f36ef08e3cb9f0b)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:19.280138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:19.294415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:19.331136 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '114ed6a689514c058f36ef08e3cb9f0b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.331828 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '114ed6a689514c058f36ef08e3cb9f0b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.332421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '114ed6a689514c058f36ef08e3cb9f0b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.343036 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '114ed6a689514c058f36ef08e3cb9f0b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.350869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-6e7da88a-06cd-44e4-ba36-bf5fb4aee3c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '114ed6a689514c058f36ef08e3cb9f0b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.367831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1382/2770] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:19 2022] DELETE /identity/v3/users/114ed6a689514c058f36ef08e3cb9f0b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:19.554965 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:19.556453 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:19.570338 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86440fd9-8967-45cd-871b-836b59c2e1c3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:19.628321 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:19.629211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:19.629490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:19.629761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] PATH_INFO: `/v3/projects/70a920cc4a634d119a0f61f4bbc567e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:19.630948 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=70a920cc4a634d119a0f61f4bbc567e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:19.632905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:19.683692 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70a920cc4a634d119a0f61f4bbc567e4', 'request_id': 'req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.691472 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70a920cc4a634d119a0f61f4bbc567e4', 'request_id': 'req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.701215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70a920cc4a634d119a0f61f4bbc567e4', 'request_id': 'req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.723986 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] The token cache is being invalidate because project 70a920cc4a634d119a0f61f4bbc567e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:19.724347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.725096 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-4ccedc5d-1919-46c9-82a0-bbd9dc5457ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:19.727021 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1389/2771] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:19 2022] DELETE /identity/v3/projects/70a920cc4a634d119a0f61f4bbc567e4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:19.732139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a22f4-a732-459b-81a1-b07aae8316c7 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:19.732603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a22f4-a732-459b-81a1-b07aae8316c7 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:19.732603 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a22f4-a732-459b-81a1-b07aae8316c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:19.752002 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ad6a22f4-a732-459b-81a1-b07aae8316c7 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:19.753280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ad6a22f4-a732-459b-81a1-b07aae8316c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:19.817315 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1383/2772] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:19.825413 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa0e05-76fd-433d-994c-7cedd6a47180 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:19.825707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa0e05-76fd-433d-994c-7cedd6a47180 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:19.825989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa0e05-76fd-433d-994c-7cedd6a47180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:19.848865 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b8aa0e05-76fd-433d-994c-7cedd6a47180 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:19.855289 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b8aa0e05-76fd-433d-994c-7cedd6a47180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:19.869682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1390/2773] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:19.874145 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a22f4-a732-459b-81a1-b07aae8316c7 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:19.884170 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a22f4-a732-459b-81a1-b07aae8316c7 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:19.893305 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:19.894055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:19.894341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:19.894622 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:19.895431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:21:19.895751 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:19.897379 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:19.909655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1384/2774] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:21:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:21:19.914587 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa0e05-76fd-433d-994c-7cedd6a47180 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:19.926229 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa0e05-76fd-433d-994c-7cedd6a47180 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:19.930916 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce15368-a4de-41ba-855f-d8e9901f46d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:19.931615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce15368-a4de-41ba-855f-d8e9901f46d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:19.931901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce15368-a4de-41ba-855f-d8e9901f46d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:19.932174 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce15368-a4de-41ba-855f-d8e9901f46d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:19.932872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce15368-a4de-41ba-855f-d8e9901f46d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:19.934794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce15368-a4de-41ba-855f-d8e9901f46d2 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:19.964379 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1391/2775] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:19 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:21:19.968924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:19.980175 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034086a6-4f9f-4bb4-aa48-0fa54305cfc3 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:20.053317 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f078ee-dedd-4a9a-b7c8-bb78c43e4937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:20.053997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f078ee-dedd-4a9a-b7c8-bb78c43e4937 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.054281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f078ee-dedd-4a9a-b7c8-bb78c43e4937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.054556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f078ee-dedd-4a9a-b7c8-bb78c43e4937 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.055269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f078ee-dedd-4a9a-b7c8-bb78c43e4937 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:20.057426 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f078ee-dedd-4a9a-b7c8-bb78c43e4937 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:20.082905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1385/2776] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:19 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:21:20.087005 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce15368-a4de-41ba-855f-d8e9901f46d2 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:20.098507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce15368-a4de-41ba-855f-d8e9901f46d2 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:20.103152 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2a7d-cb6f-4a44-94cf-80fc6b910f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:20.103850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2a7d-cb6f-4a44-94cf-80fc6b910f5d admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.104132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2a7d-cb6f-4a44-94cf-80fc6b910f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.104406 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910c2a7d-cb6f-4a44-94cf-80fc6b910f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.105141 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c2a7d-cb6f-4a44-94cf-80fc6b910f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:20.106741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910c2a7d-cb6f-4a44-94cf-80fc6b910f5d admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:20.117361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1392/2777] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:20.121805 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f078ee-dedd-4a9a-b7c8-bb78c43e4937 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:20.133325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f078ee-dedd-4a9a-b7c8-bb78c43e4937 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:20.138483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6fffc-8cdc-48d0-afc1-0b2f66a36831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:20.139128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6fffc-8cdc-48d0-afc1-0b2f66a36831 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.139457 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6fffc-8cdc-48d0-afc1-0b2f66a36831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.139736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6fffc-8cdc-48d0-afc1-0b2f66a36831 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.140879 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6fffc-8cdc-48d0-afc1-0b2f66a36831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:20.142613 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6fffc-8cdc-48d0-afc1-0b2f66a36831 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:20.144178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:20.158020 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1386/2778] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:20.162403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2a7d-cb6f-4a44-94cf-80fc6b910f5d admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:20.172492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910c2a7d-cb6f-4a44-94cf-80fc6b910f5d admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:20.178101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:20.179024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.179285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.179574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] PATH_INFO: `/v3/projects/0ecf96dacbf1471f8b8a33501e6202a8/users/9b9893bffd5b4614a70e73470607ae43/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.196052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecf96dacbf1471f8b8a33501e6202a8, user_id=9b9893bffd5b4614a70e73470607ae43, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:20.198790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:20.213809 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1393/2779] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:20 2022] PUT /identity/v3/projects/0ecf96dacbf1471f8b8a33501e6202a8/users/9b9893bffd5b4614a70e73470607ae43/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:20.224495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baae59-f007-49b8-a148-3d7f4256e648 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.224783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baae59-f007-49b8-a148-3d7f4256e648 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.225058 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41baae59-f007-49b8-a148-3d7f4256e648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.254510 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-41baae59-f007-49b8-a148-3d7f4256e648 None None] MFA Rules not processed for user `9b9893bffd5b4614a70e73470607ae43`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:20.255814 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-41baae59-f007-49b8-a148-3d7f4256e648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:20.322494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1387/2780] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:20.334460 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:20.336203 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:20.350136 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:20.396146 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b657-461d-4767-89c8-a561fea75762 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:20.399933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eea688a-8d96-4838-aaee-506a25cd6003 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:20.400575 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea688a-8d96-4838-aaee-506a25cd6003 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.400855 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea688a-8d96-4838-aaee-506a25cd6003 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.401125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eea688a-8d96-4838-aaee-506a25cd6003 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.410612 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea688a-8d96-4838-aaee-506a25cd6003 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:20.412129 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eea688a-8d96-4838-aaee-506a25cd6003 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:20.421556 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1394/2781] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:20.441736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41baae59-f007-49b8-a148-3d7f4256e648 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:20.522462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41baae59-f007-49b8-a148-3d7f4256e648 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:20.531538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:20.532522 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.532799 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.533076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] PATH_INFO: `/v3/users/9b9893bffd5b4614a70e73470607ae43` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.534494 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9893bffd5b4614a70e73470607ae43)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:20.536516 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:20.593392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9893bffd5b4614a70e73470607ae43'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:20.594006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9893bffd5b4614a70e73470607ae43'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:20.594604 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9893bffd5b4614a70e73470607ae43'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:20.603935 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9893bffd5b4614a70e73470607ae43'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:20.611465 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9893bffd5b4614a70e73470607ae43'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:20.629466 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1388/2782] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:20 2022] DELETE /identity/v3/users/9b9893bffd5b4614a70e73470607ae43 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:20.633761 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e028735-6170-481b-a0c8-5ff8e7932139 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.634051 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e028735-6170-481b-a0c8-5ff8e7932139 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.634329 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e028735-6170-481b-a0c8-5ff8e7932139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.653611 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-5e028735-6170-481b-a0c8-5ff8e7932139 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:20.654853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5e028735-6170-481b-a0c8-5ff8e7932139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:20.718064 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1395/2783] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:20.728271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:20.729889 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:20.737068 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:20.796256 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df41b31-d353-4bf0-a145-695a45728230 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:20.800027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b6da4-9bbf-456c-9d33-e1fd6fafd282 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:20.800683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6da4-9bbf-456c-9d33-e1fd6fafd282 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:20.800957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6da4-9bbf-456c-9d33-e1fd6fafd282 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:20.801234 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b6da4-9bbf-456c-9d33-e1fd6fafd282 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:20.813374 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b6da4-9bbf-456c-9d33-e1fd6fafd282 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:20.814825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b6da4-9bbf-456c-9d33-e1fd6fafd282 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:20.825943 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1389/2784] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:21.126529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e028735-6170-481b-a0c8-5ff8e7932139 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:21.132766 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-5e028735-6170-481b-a0c8-5ff8e7932139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:21.147851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e028735-6170-481b-a0c8-5ff8e7932139 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:21.159024 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:21.159968 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.160246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.160516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] PATH_INFO: `/v3/projects/0ecf96dacbf1471f8b8a33501e6202a8` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.161689 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ecf96dacbf1471f8b8a33501e6202a8)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:21.164160 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:21.222439 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ecf96dacbf1471f8b8a33501e6202a8', 'request_id': 'req-9ffb13ab-3567-4cff-a338-3357da6d41d0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:21.228751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ecf96dacbf1471f8b8a33501e6202a8', 'request_id': 'req-9ffb13ab-3567-4cff-a338-3357da6d41d0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:21.235143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ecf96dacbf1471f8b8a33501e6202a8', 'request_id': 'req-9ffb13ab-3567-4cff-a338-3357da6d41d0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:21.250591 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] The token cache is being invalidate because project 0ecf96dacbf1471f8b8a33501e6202a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:21.250881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:21.251565 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-9ffb13ab-3567-4cff-a338-3357da6d41d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:21.253171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1396/2785] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:21 2022] DELETE /identity/v3/projects/0ecf96dacbf1471f8b8a33501e6202a8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:21.258119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa923df0-297d-4995-8d86-6513528cd208 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.258424 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa923df0-297d-4995-8d86-6513528cd208 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.258883 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa923df0-297d-4995-8d86-6513528cd208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.279223 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-fa923df0-297d-4995-8d86-6513528cd208 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:21.280464 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-fa923df0-297d-4995-8d86-6513528cd208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:21.343275 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1390/2786] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:21.352221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4f819-a649-47b9-9389-056f9213d39c None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.352510 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4f819-a649-47b9-9389-056f9213d39c None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.352780 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4f819-a649-47b9-9389-056f9213d39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.372242 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-3cf4f819-a649-47b9-9389-056f9213d39c None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:21.373490 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3cf4f819-a649-47b9-9389-056f9213d39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:21.382007 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1397/2787] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:21.386539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa923df0-297d-4995-8d86-6513528cd208 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:21.397681 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa923df0-297d-4995-8d86-6513528cd208 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:21.403178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:21.403895 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.404180 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.404456 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.405198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:21:21.405508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:21.407163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:21.419467 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1391/2788] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:21:21.424059 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4f819-a649-47b9-9389-056f9213d39c None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:21.433648 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4f819-a649-47b9-9389-056f9213d39c None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:21.439312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb11888-0d6e-4c97-a799-3fd2b8e908a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:21.440826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb11888-0d6e-4c97-a799-3fd2b8e908a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.441113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb11888-0d6e-4c97-a799-3fd2b8e908a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.441708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb11888-0d6e-4c97-a799-3fd2b8e908a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.442636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb11888-0d6e-4c97-a799-3fd2b8e908a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:21.445186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb11888-0d6e-4c97-a799-3fd2b8e908a6 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:21.479298 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1398/2789] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:21 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:21.483691 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:21.494683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecee8c90-1635-4fa4-b10d-cc20680d09d1 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:21.559061 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb294ab-d801-4d49-b101-d763de4cdd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:21.559786 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb294ab-d801-4d49-b101-d763de4cdd69 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.560072 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb294ab-d801-4d49-b101-d763de4cdd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.560742 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb294ab-d801-4d49-b101-d763de4cdd69 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.561053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb294ab-d801-4d49-b101-d763de4cdd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:21.563029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb294ab-d801-4d49-b101-d763de4cdd69 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:21.587474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1392/2790] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:21 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:21.592135 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb11888-0d6e-4c97-a799-3fd2b8e908a6 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:21.603847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb11888-0d6e-4c97-a799-3fd2b8e908a6 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:21.609388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87124dad-ae71-4ccc-86bb-dd6da50aa41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:21.610063 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87124dad-ae71-4ccc-86bb-dd6da50aa41b admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.610348 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87124dad-ae71-4ccc-86bb-dd6da50aa41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.610626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87124dad-ae71-4ccc-86bb-dd6da50aa41b admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.611402 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87124dad-ae71-4ccc-86bb-dd6da50aa41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:21.613054 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87124dad-ae71-4ccc-86bb-dd6da50aa41b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:21.615554 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:21.626704 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1399/2791] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:21.630755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb294ab-d801-4d49-b101-d763de4cdd69 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:21.644302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb294ab-d801-4d49-b101-d763de4cdd69 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:21.649712 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea579a-fbf6-4471-b921-a9e5516dd73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:21.650401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea579a-fbf6-4471-b921-a9e5516dd73e admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.650686 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea579a-fbf6-4471-b921-a9e5516dd73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.650964 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea579a-fbf6-4471-b921-a9e5516dd73e admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.651755 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea579a-fbf6-4471-b921-a9e5516dd73e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:21.653392 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea579a-fbf6-4471-b921-a9e5516dd73e admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:21.666067 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1393/2792] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:21.670628 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87124dad-ae71-4ccc-86bb-dd6da50aa41b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:21.681508 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87124dad-ae71-4ccc-86bb-dd6da50aa41b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:21.687091 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:21.688265 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.688514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.688789 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] PATH_INFO: `/v3/projects/db30a18e05104c86afd41edf5aca846c/users/eca6d6b5939146fa97c67b8188e329c1/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.706903 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db30a18e05104c86afd41edf5aca846c, user_id=eca6d6b5939146fa97c67b8188e329c1, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:21.709626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:21.725320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1400/2793] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:21 2022] PUT /identity/v3/projects/db30a18e05104c86afd41edf5aca846c/users/eca6d6b5939146fa97c67b8188e329c1/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:21.735685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ec513-61c6-488e-8610-5713c348295d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.736003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ec513-61c6-488e-8610-5713c348295d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.736285 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ec513-61c6-488e-8610-5713c348295d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.762521 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:21.766089 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-914ec513-61c6-488e-8610-5713c348295d None None] MFA Rules not processed for user `eca6d6b5939146fa97c67b8188e329c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:21.767984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-914ec513-61c6-488e-8610-5713c348295d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:21.840660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1394/2794] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:21.851557 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:21.853372 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:21.918070 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e063324-1280-4a75-bf19-7e25090155f4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:21.921697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cee69a-99de-4429-978b-68ba8a3ab63d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:21.922429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cee69a-99de-4429-978b-68ba8a3ab63d service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:21.922620 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cee69a-99de-4429-978b-68ba8a3ab63d service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:21.922928 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cee69a-99de-4429-978b-68ba8a3ab63d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:21.945980 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cee69a-99de-4429-978b-68ba8a3ab63d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:21.947666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cee69a-99de-4429-978b-68ba8a3ab63d service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:21.958683 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1401/2795] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:31.837454 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=26 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:31.954215 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:37.403415 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ec513-61c6-488e-8610-5713c348295d None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:37.470540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ec513-61c6-488e-8610-5713c348295d None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:37.475010 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:37.475959 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:37.476203 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:37.476478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] PATH_INFO: `/v3/users/eca6d6b5939146fa97c67b8188e329c1` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:37.477685 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eca6d6b5939146fa97c67b8188e329c1)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:37.479677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:37.530651 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eca6d6b5939146fa97c67b8188e329c1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:37.531471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eca6d6b5939146fa97c67b8188e329c1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:37.532084 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eca6d6b5939146fa97c67b8188e329c1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:37.540304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eca6d6b5939146fa97c67b8188e329c1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:37.547573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eca6d6b5939146fa97c67b8188e329c1'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:37.565014 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1395/2796] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:37 2022] DELETE /identity/v3/users/eca6d6b5939146fa97c67b8188e329c1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:37.570121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19298835-aaff-4c1e-a57f-342dc15fc863 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:37.570605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19298835-aaff-4c1e-a57f-342dc15fc863 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:37.570727 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19298835-aaff-4c1e-a57f-342dc15fc863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:37.599416 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-19298835-aaff-4c1e-a57f-342dc15fc863 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:37.600729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-19298835-aaff-4c1e-a57f-342dc15fc863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:37.678651 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1402/2797] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:37.689147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:37.690612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:37.749630 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b83fdb0-c7b3-404e-9b7d-c90eecfb66a5 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:37.753513 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863171c-24e8-43d1-810f-f4234a985795 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:37.754153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863171c-24e8-43d1-810f-f4234a985795 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:37.754480 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863171c-24e8-43d1-810f-f4234a985795 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:37.754752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863171c-24e8-43d1-810f-f4234a985795 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:37.763856 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863171c-24e8-43d1-810f-f4234a985795 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:37.765644 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863171c-24e8-43d1-810f-f4234a985795 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:37.775836 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1396/2798] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:38.054249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19298835-aaff-4c1e-a57f-342dc15fc863 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:38.055840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-19298835-aaff-4c1e-a57f-342dc15fc863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:38.070442 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19298835-aaff-4c1e-a57f-342dc15fc863 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:38.079052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:38.079847 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.080134 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.080399 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] PATH_INFO: `/v3/projects/db30a18e05104c86afd41edf5aca846c` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.081790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db30a18e05104c86afd41edf5aca846c)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:38.083601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:38.152786 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db30a18e05104c86afd41edf5aca846c', 'request_id': 'req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:38.159297 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db30a18e05104c86afd41edf5aca846c', 'request_id': 'req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:38.165770 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db30a18e05104c86afd41edf5aca846c', 'request_id': 'req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:38.179579 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] The token cache is being invalidate because project db30a18e05104c86afd41edf5aca846c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:38.179868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:38.180482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f26a22e0-9a11-42ff-b2a4-2e1c4f9dead0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:38.182039 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1403/2799] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:38 2022] DELETE /identity/v3/projects/db30a18e05104c86afd41edf5aca846c => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:38.187205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c198f-fc4a-429c-afe4-c9d3b9db3052 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.187515 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c198f-fc4a-429c-afe4-c9d3b9db3052 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.187792 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c198f-fc4a-429c-afe4-c9d3b9db3052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.205671 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-3a9c198f-fc4a-429c-afe4-c9d3b9db3052 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:38.206866 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-3a9c198f-fc4a-429c-afe4-c9d3b9db3052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:38.222831 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:38.265979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1397/2800] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:38.275071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa8d52-1735-4169-abfd-a2602e4a93f1 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.275396 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa8d52-1735-4169-abfd-a2602e4a93f1 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.275676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8aa8d52-1735-4169-abfd-a2602e4a93f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.294404 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f8aa8d52-1735-4169-abfd-a2602e4a93f1 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:38.295643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f8aa8d52-1735-4169-abfd-a2602e4a93f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:38.303870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1404/2801] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:38.308286 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c198f-fc4a-429c-afe4-c9d3b9db3052 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:38.318356 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9c198f-fc4a-429c-afe4-c9d3b9db3052 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:38.323901 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:38.324744 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.325138 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.325437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.326176 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:21:38.326479 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:38.328210 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:38.340102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1398/2802] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:21:38.344282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa8d52-1735-4169-abfd-a2602e4a93f1 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:38.353729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8aa8d52-1735-4169-abfd-a2602e4a93f1 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:38.358794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2442-07d0-4b13-9ece-0827645272c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:38.359476 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2442-07d0-4b13-9ece-0827645272c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.359760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2442-07d0-4b13-9ece-0827645272c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.360040 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be2442-07d0-4b13-9ece-0827645272c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.360732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be2442-07d0-4b13-9ece-0827645272c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:38.362623 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be2442-07d0-4b13-9ece-0827645272c4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:38.390997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1405/2803] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:38 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:38.395177 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:38.405584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f59d19-69ba-4d1d-a402-d49bba6332d0 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:38.467198 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0c118-a1ab-49e8-ab61-ecbeb1af8149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:38.468163 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0c118-a1ab-49e8-ab61-ecbeb1af8149 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.468540 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0c118-a1ab-49e8-ab61-ecbeb1af8149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.468936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0c118-a1ab-49e8-ab61-ecbeb1af8149 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.469961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0c118-a1ab-49e8-ab61-ecbeb1af8149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:38.473152 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0c118-a1ab-49e8-ab61-ecbeb1af8149 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:38.499104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1399/2804] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:38 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:38.503356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2442-07d0-4b13-9ece-0827645272c4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:38.514132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be2442-07d0-4b13-9ece-0827645272c4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:38.519738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abb27b-a0ec-44d4-a945-d382da7c98c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:38.520430 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb27b-a0ec-44d4-a945-d382da7c98c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.520713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb27b-a0ec-44d4-a945-d382da7c98c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.521003 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abb27b-a0ec-44d4-a945-d382da7c98c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.521728 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abb27b-a0ec-44d4-a945-d382da7c98c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:38.523377 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2abb27b-a0ec-44d4-a945-d382da7c98c1 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:38.533838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1406/2805] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:38.537983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0c118-a1ab-49e8-ab61-ecbeb1af8149 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:38.549488 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0c118-a1ab-49e8-ab61-ecbeb1af8149 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:38.555251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8166167-0878-45be-a0c3-a9ba9c819526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:38.556076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8166167-0878-45be-a0c3-a9ba9c819526 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.556359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8166167-0878-45be-a0c3-a9ba9c819526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.556673 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8166167-0878-45be-a0c3-a9ba9c819526 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.557481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8166167-0878-45be-a0c3-a9ba9c819526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:38.559508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8166167-0878-45be-a0c3-a9ba9c819526 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:38.570222 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1400/2806] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:38.574755 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abb27b-a0ec-44d4-a945-d382da7c98c1 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:38.585302 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abb27b-a0ec-44d4-a945-d382da7c98c1 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:38.590729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:38.591615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.591893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.592161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] PATH_INFO: `/v3/projects/aabdb72f128d4d4da1f9390e505d704b/users/7abef7e1523046a8970d7010ae02c37e/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.607055 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabdb72f128d4d4da1f9390e505d704b, user_id=7abef7e1523046a8970d7010ae02c37e, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:38.609652 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:38.624334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1407/2807] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:38 2022] PUT /identity/v3/projects/aabdb72f128d4d4da1f9390e505d704b/users/7abef7e1523046a8970d7010ae02c37e/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:38.629413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495fa93-1117-4e47-aa14-d76c4c83139d None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.629725 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495fa93-1117-4e47-aa14-d76c4c83139d None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.630005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495fa93-1117-4e47-aa14-d76c4c83139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.652266 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-4495fa93-1117-4e47-aa14-d76c4c83139d None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:38.653553 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4495fa93-1117-4e47-aa14-d76c4c83139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:38.716783 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1401/2808] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:38.723632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:38.725428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:38.747223 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:38.795281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773cc7b-7149-467c-9678-20876c27da36 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:38.799389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138c22c-5eff-4b69-99db-6bf2e883c1a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:38.800076 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138c22c-5eff-4b69-99db-6bf2e883c1a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:38.800389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138c22c-5eff-4b69-99db-6bf2e883c1a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:38.800708 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5138c22c-5eff-4b69-99db-6bf2e883c1a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:38.810352 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138c22c-5eff-4b69-99db-6bf2e883c1a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:38.811944 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5138c22c-5eff-4b69-99db-6bf2e883c1a0 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:38.822823 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1408/2809] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:42.776580 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de06c6-eca1-4914-b184-98bf0c7beb38 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:42.777098 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de06c6-eca1-4914-b184-98bf0c7beb38 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:42.777222 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77de06c6-eca1-4914-b184-98bf0c7beb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:42.808806 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-77de06c6-eca1-4914-b184-98bf0c7beb38 None None] MFA Rules not processed for user `7abef7e1523046a8970d7010ae02c37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:42.810108 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-77de06c6-eca1-4914-b184-98bf0c7beb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:42.829983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:42.890596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1402/2810] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:42 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:42.897853 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138c22c-5eff-4b69-99db-6bf2e883c1a0 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:42.908014 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5138c22c-5eff-4b69-99db-6bf2e883c1a0 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:42.912933 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c88ba6-9fd5-4081-8312-b115e0247176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:42.913675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c88ba6-9fd5-4081-8312-b115e0247176 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:42.914026 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c88ba6-9fd5-4081-8312-b115e0247176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:42.914358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c88ba6-9fd5-4081-8312-b115e0247176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:42.915113 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c88ba6-9fd5-4081-8312-b115e0247176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:42.917117 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c88ba6-9fd5-4081-8312-b115e0247176 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:42.945562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1409/2811] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:42 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:42.949699 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de06c6-eca1-4914-b184-98bf0c7beb38 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:42.960867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77de06c6-eca1-4914-b184-98bf0c7beb38 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:43.017419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9299f82-b395-4f39-8738-22110e82e6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:43.018105 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9299f82-b395-4f39-8738-22110e82e6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:43.018387 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9299f82-b395-4f39-8738-22110e82e6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:43.018662 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9299f82-b395-4f39-8738-22110e82e6d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:43.019527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9299f82-b395-4f39-8738-22110e82e6d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:43.021431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9299f82-b395-4f39-8738-22110e82e6d9 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:43.050322 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1403/2812] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:42 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:21:43.055495 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c88ba6-9fd5-4081-8312-b115e0247176 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:43.070793 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c88ba6-9fd5-4081-8312-b115e0247176 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:43.086445 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5bd7b-cfd6-403f-9183-0f5bceb060c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:43.087511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5bd7b-cfd6-403f-9183-0f5bceb060c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:43.087895 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5bd7b-cfd6-403f-9183-0f5bceb060c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:43.088186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c5bd7b-cfd6-403f-9183-0f5bceb060c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:43.089000 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5bd7b-cfd6-403f-9183-0f5bceb060c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:43.090818 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c5bd7b-cfd6-403f-9183-0f5bceb060c3 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:43.102331 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1410/2813] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:43.106487 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9299f82-b395-4f39-8738-22110e82e6d9 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:43.119537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9299f82-b395-4f39-8738-22110e82e6d9 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:43.126650 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030c4c1-311b-4cae-9545-81cb9a619e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:43.127683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030c4c1-311b-4cae-9545-81cb9a619e43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:43.127971 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030c4c1-311b-4cae-9545-81cb9a619e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:43.128249 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3030c4c1-311b-4cae-9545-81cb9a619e43 admin admin] PATH_INFO: `/v3/projects/50db3d6e2f174362b1cf5e3d3e11eb2b/users/0c97a09c9d414424bfa71a0c78110671/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:43.145663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030c4c1-311b-4cae-9545-81cb9a619e43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50db3d6e2f174362b1cf5e3d3e11eb2b, user_id=0c97a09c9d414424bfa71a0c78110671, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:43.148398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3030c4c1-311b-4cae-9545-81cb9a619e43 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:43.169368 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1404/2814] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:43 2022] PUT /identity/v3/projects/50db3d6e2f174362b1cf5e3d3e11eb2b/users/0c97a09c9d414424bfa71a0c78110671/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:43.173166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5bd7b-cfd6-403f-9183-0f5bceb060c3 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:43.184048 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c5bd7b-cfd6-403f-9183-0f5bceb060c3 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:43.251738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68e62e9-0809-4142-b134-0e0f6cfba7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:43.252429 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e62e9-0809-4142-b134-0e0f6cfba7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:43.252991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e62e9-0809-4142-b134-0e0f6cfba7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:43.252991 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68e62e9-0809-4142-b134-0e0f6cfba7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:43.253709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e62e9-0809-4142-b134-0e0f6cfba7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:43.255726 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68e62e9-0809-4142-b134-0e0f6cfba7c9 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:43.273080 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1411/2815] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:43.276944 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030c4c1-311b-4cae-9545-81cb9a619e43 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:43.290045 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3030c4c1-311b-4cae-9545-81cb9a619e43 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:43.297938 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:43.298967 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:43.299557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:43.300318 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] PATH_INFO: `/v3/projects/50db3d6e2f174362b1cf5e3d3e11eb2b/users/0c97a09c9d414424bfa71a0c78110671/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:43.308106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=50db3d6e2f174362b1cf5e3d3e11eb2b, user_id=0c97a09c9d414424bfa71a0c78110671, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:43.319934 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:43.332308 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1405/2816] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:43 2022] PUT /identity/v3/projects/50db3d6e2f174362b1cf5e3d3e11eb2b/users/0c97a09c9d414424bfa71a0c78110671/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:48.074851 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f472cd3-e404-4f13-9005-cb04f0e23b25 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:48.075388 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f472cd3-e404-4f13-9005-cb04f0e23b25 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:48.075493 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f472cd3-e404-4f13-9005-cb04f0e23b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:48.108675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-8f472cd3-e404-4f13-9005-cb04f0e23b25 None None] MFA Rules not processed for user `0c97a09c9d414424bfa71a0c78110671`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:48.110238 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-8f472cd3-e404-4f13-9005-cb04f0e23b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:48.189259 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1412/2817] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:48.207091 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:48.209154 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:48.291584 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5393623b-9459-40cd-a3d2-9104535c74bc admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:48.295401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '68d25a45f9ee4ae58c6751a8781d12d1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:48.296064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:48.296350 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:48.296629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:48.322201 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:48.374665 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:48.376776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:48.407401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1406/2818] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:48.513111 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f174c4-4b95-4762-bb06-b64c568c75de None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:48.513450 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f174c4-4b95-4762-bb06-b64c568c75de None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:48.513733 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f174c4-4b95-4762-bb06-b64c568c75de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:48.533052 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-b6f174c4-4b95-4762-bb06-b64c568c75de None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:48.534324 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b6f174c4-4b95-4762-bb06-b64c568c75de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:48.540071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:48.603966 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1413/2819] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:21:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:48.671987 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:48.673431 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:48.763436 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974f1fa-d039-42ad-8b30-82f7973ec2fe service glance] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:48.768900 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8eb29d-48ac-49c9-a401-c2e30422d4df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:48.769583 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb29d-48ac-49c9-a401-c2e30422d4df service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:48.769872 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb29d-48ac-49c9-a401-c2e30422d4df service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:48.770150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8eb29d-48ac-49c9-a401-c2e30422d4df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:48.781531 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8eb29d-48ac-49c9-a401-c2e30422d4df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:48.783902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8eb29d-48ac-49c9-a401-c2e30422d4df service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:48.798114 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1407/2820] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:21:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:21:49.782943 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f174c4-4b95-4762-bb06-b64c568c75de None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:49.784408 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b6f174c4-4b95-4762-bb06-b64c568c75de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:49.890826 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f174c4-4b95-4762-bb06-b64c568c75de None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:49.896159 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:49.897350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:49.897873 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:49.898609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] PATH_INFO: `/v3/users/5e591d6211504fb4a2a54e260c2f117a` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:49.900165 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e591d6211504fb4a2a54e260c2f117a)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:49.904609 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:50.004572 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e591d6211504fb4a2a54e260c2f117a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.012463 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e591d6211504fb4a2a54e260c2f117a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.013180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e591d6211504fb4a2a54e260c2f117a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.030284 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e591d6211504fb4a2a54e260c2f117a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.031143 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e591d6211504fb4a2a54e260c2f117a'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.074197 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1414/2821] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:49 2022] DELETE /identity/v3/users/5e591d6211504fb4a2a54e260c2f117a => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:50.078541 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd9670-e172-464c-8358-09e3b55eabd0 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:50.079097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd9670-e172-464c-8358-09e3b55eabd0 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:50.079097 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcd9670-e172-464c-8358-09e3b55eabd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:50.110760 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-afcd9670-e172-464c-8358-09e3b55eabd0 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:50.112024 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-afcd9670-e172-464c-8358-09e3b55eabd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:50.227699 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:50.229219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:50.234839 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1408/2822] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:50.245088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd9670-e172-464c-8358-09e3b55eabd0 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:50.248732 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-afcd9670-e172-464c-8358-09e3b55eabd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:50.257501 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18945fcb-f0a7-4113-9d88-9312fd7d538b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:50.267936 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:50.269120 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:50.269684 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:50.270128 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] PATH_INFO: `/v3/users/661271a657a548468ff85fe298e0c5f9` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:50.271624 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] RBAC: Authorizing `identity:delete_user(user_id=661271a657a548468ff85fe298e0c5f9)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:50.273729 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:50.333363 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:50.357989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661271a657a548468ff85fe298e0c5f9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.361569 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcd9670-e172-464c-8358-09e3b55eabd0 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:50.368996 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661271a657a548468ff85fe298e0c5f9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.369716 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661271a657a548468ff85fe298e0c5f9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.370991 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3070b7f-8814-4b2d-a4b0-d53206080490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:50.372021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3070b7f-8814-4b2d-a4b0-d53206080490 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:50.372483 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3070b7f-8814-4b2d-a4b0-d53206080490 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:50.373078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3070b7f-8814-4b2d-a4b0-d53206080490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:50.382757 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661271a657a548468ff85fe298e0c5f9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.383504 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661271a657a548468ff85fe298e0c5f9'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.390385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3070b7f-8814-4b2d-a4b0-d53206080490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:50.392269 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3070b7f-8814-4b2d-a4b0-d53206080490 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:50.399664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=100 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:50.407422 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1409/2823] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:50.414931 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1415/2824] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:50 2022] DELETE /identity/v3/users/661271a657a548468ff85fe298e0c5f9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:50.696496 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3070b7f-8814-4b2d-a4b0-d53206080490 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:50.708144 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3070b7f-8814-4b2d-a4b0-d53206080490 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:50.808888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:50.810137 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:50.810570 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:50.811011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] PATH_INFO: `/v3/projects/e619f7d90e8543ba85f3bbaba9046c72` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:50.814826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e619f7d90e8543ba85f3bbaba9046c72)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:50.817251 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:50.842868 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:50.846262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:50.883123 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e619f7d90e8543ba85f3bbaba9046c72', 'request_id': 'req-32dcdfea-9c67-4271-8ff5-f7821794808b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.892670 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a182d2-c5c5-44c7-8cc8-5f30caffe38f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:50.897351 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e619f7d90e8543ba85f3bbaba9046c72', 'request_id': 'req-32dcdfea-9c67-4271-8ff5-f7821794808b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.908106 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e619f7d90e8543ba85f3bbaba9046c72', 'request_id': 'req-32dcdfea-9c67-4271-8ff5-f7821794808b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.927663 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] The token cache is being invalidate because project e619f7d90e8543ba85f3bbaba9046c72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:50.928423 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.930117 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-32dcdfea-9c67-4271-8ff5-f7821794808b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:50.931919 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1410/2825] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:50 2022] DELETE /identity/v3/projects/e619f7d90e8543ba85f3bbaba9046c72 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:50.970282 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:50.971187 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:50.971511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:50.971779 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] PATH_INFO: `/v3/projects/5eeca6a310014d5c881a144f59a5f329` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:50.973178 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eeca6a310014d5c881a144f59a5f329)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:50.975161 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:51.056535 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eeca6a310014d5c881a144f59a5f329', 'request_id': 'req-f929d053-c1d9-4d0e-bb0b-206649824d9c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:51.067057 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eeca6a310014d5c881a144f59a5f329', 'request_id': 'req-f929d053-c1d9-4d0e-bb0b-206649824d9c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:51.087090 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eeca6a310014d5c881a144f59a5f329', 'request_id': 'req-f929d053-c1d9-4d0e-bb0b-206649824d9c'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:51.131951 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] The token cache is being invalidate because project 5eeca6a310014d5c881a144f59a5f329 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:51.132339 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:51.133085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f929d053-c1d9-4d0e-bb0b-206649824d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:51.138722 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1416/2826] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:50 2022] DELETE /identity/v3/projects/5eeca6a310014d5c881a144f59a5f329 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:21:51.144983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29076d-64bf-4ff5-bc90-af16033bfb1a None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.147501 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29076d-64bf-4ff5-bc90-af16033bfb1a None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.148129 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad29076d-64bf-4ff5-bc90-af16033bfb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.177402 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-ad29076d-64bf-4ff5-bc90-af16033bfb1a None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:51.183396 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ad29076d-64bf-4ff5-bc90-af16033bfb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:51.293826 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1411/2827] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:51.301036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669121-a969-41fb-ab58-500041380b33 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.303278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669121-a969-41fb-ab58-500041380b33 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.303278 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77669121-a969-41fb-ab58-500041380b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.324878 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-77669121-a969-41fb-ab58-500041380b33 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:51.326364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-77669121-a969-41fb-ab58-500041380b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:51.336511 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1417/2828] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:51.340419 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29076d-64bf-4ff5-bc90-af16033bfb1a None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:51.353099 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad29076d-64bf-4ff5-bc90-af16033bfb1a None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:51.361710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:51.362413 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.362703 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.362984 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.364076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:21:51.364391 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:51.366490 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:51.385968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1412/2829] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:21:51.390682 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77669121-a969-41fb-ab58-500041380b33 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:51.402525 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77669121-a969-41fb-ab58-500041380b33 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:51.408573 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c17a-d335-4ebd-bdc0-bcc5faad295b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:51.409367 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c17a-d335-4ebd-bdc0-bcc5faad295b admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.409658 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c17a-d335-4ebd-bdc0-bcc5faad295b admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.409934 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b7c17a-d335-4ebd-bdc0-bcc5faad295b admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.410649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7c17a-d335-4ebd-bdc0-bcc5faad295b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:51.412637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b7c17a-d335-4ebd-bdc0-bcc5faad295b admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:51.460475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1418/2830] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:21:51 2022] POST /identity/v3/projects => generated 371 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:51.461679 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:51.474615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2bb9d-9d95-4fbb-8b79-81a6f1b71b56 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:51.491561 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:51.591369 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2b12b-21e6-4f2c-8f4a-5b8631f33882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:51.592069 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2b12b-21e6-4f2c-8f4a-5b8631f33882 admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.592362 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2b12b-21e6-4f2c-8f4a-5b8631f33882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.592637 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2b12b-21e6-4f2c-8f4a-5b8631f33882 admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.593416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2b12b-21e6-4f2c-8f4a-5b8631f33882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:51.595572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d2b12b-21e6-4f2c-8f4a-5b8631f33882 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:51.626331 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1413/2831] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:21:51 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:21:51.630281 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c17a-d335-4ebd-bdc0-bcc5faad295b admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:51.647911 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b7c17a-d335-4ebd-bdc0-bcc5faad295b admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:51.655907 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb8f33-7abd-44f5-8cbf-0f5847678d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:51.658649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f33-7abd-44f5-8cbf-0f5847678d32 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.659125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f33-7abd-44f5-8cbf-0f5847678d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.659561 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdb8f33-7abd-44f5-8cbf-0f5847678d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.660513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb8f33-7abd-44f5-8cbf-0f5847678d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:51.662262 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdb8f33-7abd-44f5-8cbf-0f5847678d32 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:51.666391 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:51.682441 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1419/2832] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:51.683816 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2b12b-21e6-4f2c-8f4a-5b8631f33882 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:51.699481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d2b12b-21e6-4f2c-8f4a-5b8631f33882 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:51.705629 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a00a27-cc2b-4269-8d9e-4f45fc441715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:51.706474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a00a27-cc2b-4269-8d9e-4f45fc441715 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.706827 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a00a27-cc2b-4269-8d9e-4f45fc441715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.707155 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a00a27-cc2b-4269-8d9e-4f45fc441715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.708006 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a00a27-cc2b-4269-8d9e-4f45fc441715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:51.709865 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a00a27-cc2b-4269-8d9e-4f45fc441715 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:51.723293 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1414/2833] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:21:51.726593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb8f33-7abd-44f5-8cbf-0f5847678d32 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:51.742020 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdb8f33-7abd-44f5-8cbf-0f5847678d32 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:51.748011 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:51.749183 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.749665 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.750119 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] PATH_INFO: `/v3/projects/da35131c1ab64338b0bbebea07d76f95/users/00eb096e450e41518afe12ac1a61019b/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.774505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da35131c1ab64338b0bbebea07d76f95, user_id=00eb096e450e41518afe12ac1a61019b, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:51.780137 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:51.802564 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1420/2834] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:21:51 2022] PUT /identity/v3/projects/da35131c1ab64338b0bbebea07d76f95/users/00eb096e450e41518afe12ac1a61019b/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:51.814202 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e112d7-446a-4442-898e-ff29e54b6ad1 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:51.814474 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e112d7-446a-4442-898e-ff29e54b6ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:51.814689 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e112d7-446a-4442-898e-ff29e54b6ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:51.847983 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-48e112d7-446a-4442-898e-ff29e54b6ad1 None None] MFA Rules not processed for user `00eb096e450e41518afe12ac1a61019b`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:51.849213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-48e112d7-446a-4442-898e-ff29e54b6ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:51.942213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1415/2835] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:51.954486 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:51.956608 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:52.053915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64811878-2ecd-4c76-8ce1-1bf9b2cc88d6 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:52.059220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07528f83-ea6e-4780-9dda-69f01a301e02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:52.059960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07528f83-ea6e-4780-9dda-69f01a301e02 service cinder] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:52.060244 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07528f83-ea6e-4780-9dda-69f01a301e02 service cinder] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:52.060503 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07528f83-ea6e-4780-9dda-69f01a301e02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:52.076484 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07528f83-ea6e-4780-9dda-69f01a301e02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:52.077110 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07528f83-ea6e-4780-9dda-69f01a301e02 service cinder] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:52.103480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1421/2836] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:52.118736 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e112d7-446a-4442-898e-ff29e54b6ad1 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:52.208302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e112d7-446a-4442-898e-ff29e54b6ad1 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:52.215693 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:52.216576 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:52.216902 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:52.217253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] PATH_INFO: `/v3/users/00eb096e450e41518afe12ac1a61019b` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:52.218581 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=00eb096e450e41518afe12ac1a61019b)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:52.220741 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:52.256323 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:52.286207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00eb096e450e41518afe12ac1a61019b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:52.289281 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00eb096e450e41518afe12ac1a61019b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:52.290128 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00eb096e450e41518afe12ac1a61019b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:52.304945 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00eb096e450e41518afe12ac1a61019b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:52.313720 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00eb096e450e41518afe12ac1a61019b'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:52.337999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1416/2837] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:21:52 2022] DELETE /identity/v3/users/00eb096e450e41518afe12ac1a61019b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:52.342574 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbb47a-29ad-4db6-ae43-e7adca687911 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:52.342870 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbb47a-29ad-4db6-ae43-e7adca687911 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:52.343155 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbb47a-29ad-4db6-ae43-e7adca687911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:52.366420 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-21dbb47a-29ad-4db6-ae43-e7adca687911 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:52.367702 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-21dbb47a-29ad-4db6-ae43-e7adca687911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:52.458819 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1422/2838] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:21:52.468869 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:52.471005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:52.543071 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eed075-ed12-4e3e-b5c5-a5d472f547dc admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:52.547558 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:52.548608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:52.549076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:52.549359 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:52.561715 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:52.563288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:52.588442 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1417/2839] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:52.962834 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbb47a-29ad-4db6-ae43-e7adca687911 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:52.964327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-21dbb47a-29ad-4db6-ae43-e7adca687911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:52.983626 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbb47a-29ad-4db6-ae43-e7adca687911 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:52.988751 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:52.989776 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:52.989913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:52.990551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] PATH_INFO: `/v3/projects/da35131c1ab64338b0bbebea07d76f95` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:52.991454 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=da35131c1ab64338b0bbebea07d76f95)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:52.993403 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:53.087358 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da35131c1ab64338b0bbebea07d76f95', 'request_id': 'req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:53.094741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da35131c1ab64338b0bbebea07d76f95', 'request_id': 'req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:53.103393 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da35131c1ab64338b0bbebea07d76f95', 'request_id': 'req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:53.120679 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:53.132272 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] The token cache is being invalidate because project da35131c1ab64338b0bbebea07d76f95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:21:53.132897 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:53.133956 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-a4ec6652-86e5-47dc-a5d8-2fa1580e97bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:21:53.143364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1423/2840] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:21:52 2022] DELETE /identity/v3/projects/da35131c1ab64338b0bbebea07d76f95 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:21:56.816587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:21:56.818011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:56.881003 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de5c5b1-dd55-49c0-bdc5-b8ff03f64b12 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:21:56.942538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b672a2898b849338844b1c8e911646d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:21:56.943310 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:56.943602 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:56.943912 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:56.946507 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:56.967078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:21:57.025791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:21:57.027503 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:21:57.039341 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1418/2841] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:21:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:21:58.609285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365783a-5794-4c5a-80e1-7238d320755b None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:21:58.609850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365783a-5794-4c5a-80e1-7238d320755b None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:21:58.609850 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365783a-5794-4c5a-80e1-7238d320755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:21:58.637517 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2365783a-5794-4c5a-80e1-7238d320755b None None] MFA Rules not processed for user `7abef7e1523046a8970d7010ae02c37e`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:21:58.638713 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2365783a-5794-4c5a-80e1-7238d320755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:21:58.693803 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1424/2842] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:21:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:22:00.051881 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:22:00.053409 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:22:00.125448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff0cd4-91a4-4dd5-a9e8-0077f93f213c service cinder] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:22:00.130999 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:22:00.131683 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:22:00.131964 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:22:00.132241 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:22:00.134385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:22:00.147064 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:22:00.148702 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:22:00.158766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1419/2843] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:22:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:22:00.993125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524301e-aa28-4631-9f42-0748b496499e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:22:00.993636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524301e-aa28-4631-9f42-0748b496499e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:22:00.993636 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524301e-aa28-4631-9f42-0748b496499e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:22:01.013762 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-6524301e-aa28-4631-9f42-0748b496499e None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:22:01.015370 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-6524301e-aa28-4631-9f42-0748b496499e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:22:01.072709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1425/2844] 10.222.0.69 () {58 vars in 1009 bytes} [Fri May 20 09:22:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:22:01.085963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:22:01.087407 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:22:01.159929 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2821eb27-5e4f-4b74-a0dc-3075d2e7da8a service placement] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:22:01.165772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:22:01.166435 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:22:01.166717 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:22:01.166990 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:22:01.176057 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:22:01.177668 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:22:01.185950 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1420/2845] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:22:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:22:03.955470 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1986b06-01d1-4434-b62b-31a43ab3d5eb None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:22:03.956016 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1986b06-01d1-4434-b62b-31a43ab3d5eb None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:22:03.956505 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1986b06-01d1-4434-b62b-31a43ab3d5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:22:03.994482 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-c1986b06-01d1-4434-b62b-31a43ab3d5eb None None] MFA Rules not processed for user `bc0307909d614a1f97f006dc6d091cd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:22:03.995857 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c1986b06-01d1-4434-b62b-31a43ab3d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:22:04.015982 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1426/2846] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:22:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:22:11.182801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=84 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:22:13.980615 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=53 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:22:27.917306 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:22:27.919053 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:22:27.976455 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ced58a-6733-4b12-8bdb-8c1e05f4133c service swift] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:22:27.980385 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:22:27.981073 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:22:27.981340 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:22:27.981597 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:22:27.983468 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:22:28.047278 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:22:28.048926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:22:28.059711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1421/2847] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:22:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:22:38.054818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=50 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:23:39.087427 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1986b06-01d1-4434-b62b-31a43ab3d5eb None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:39.089106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-c1986b06-01d1-4434-b62b-31a43ab3d5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:39.102700 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1986b06-01d1-4434-b62b-31a43ab3d5eb None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:39.107513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d608a-aa0d-43b0-a87d-400552c01dcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:39.108156 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d608a-aa0d-43b0-a87d-400552c01dcc service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:39.108446 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d608a-aa0d-43b0-a87d-400552c01dcc service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:39.108712 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d608a-aa0d-43b0-a87d-400552c01dcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:39.110915 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-e57d608a-aa0d-43b0-a87d-400552c01dcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:39.189452 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d608a-aa0d-43b0-a87d-400552c01dcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:39.190919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d608a-aa0d-43b0-a87d-400552c01dcc service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:39.202073 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1427/2848] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:23:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:23:45.262473 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:45.272094 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871378e-399d-4142-a030-bd314be2630d service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:45.276638 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:45.277279 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:45.277557 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:45.277825 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:45.279982 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:45.296824 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:45.298304 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:45.310008 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1422/2849] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:23:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:23:47.676874 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7117f0-6699-44e5-8a7e-16a839c9dc2d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:47.677313 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7117f0-6699-44e5-8a7e-16a839c9dc2d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:47.677432 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7117f0-6699-44e5-8a7e-16a839c9dc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:47.695637 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2a7117f0-6699-44e5-8a7e-16a839c9dc2d None None] MFA Rules not processed for user `6cd71c35c4224658aa2ab0b81f0f4b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:23:47.696901 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2a7117f0-6699-44e5-8a7e-16a839c9dc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:47.704479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1428/2850] 10.222.0.69 () {58 vars in 967 bytes} [Fri May 20 09:23:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:23:47.744166 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:47.756438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7948310c-b0b7-43f7-adb5-f1005e9dc263 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:47.762332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0c8203a8a7464c3bac93e5d42808c00f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:47.762978 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:47.763636 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:47.763904 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:47.772801 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:47.775364 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:47.785833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1423/2851] 10.222.0.69 () {60 vars in 1384 bytes} [Fri May 20 09:23:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 20 09:23:51.218881 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7117f0-6699-44e5-8a7e-16a839c9dc2d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:51.233529 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7117f0-6699-44e5-8a7e-16a839c9dc2d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:51.238859 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:51.239644 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:51.240027 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:51.240361 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:51.242234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:51.254545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:51.256347 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:51.269409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1429/2852] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:23:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:23:54.506867 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:54.508446 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:54.575907 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637461c-d359-4508-a15b-ec0b8275eca6 service swift] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:54.581199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:54.582088 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:54.582382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:54.582652 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] PATH_INFO: `/v3/users/7abef7e1523046a8970d7010ae02c37e` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:54.584087 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7abef7e1523046a8970d7010ae02c37e)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:54.586080 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:54.636366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7abef7e1523046a8970d7010ae02c37e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:54.636966 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7abef7e1523046a8970d7010ae02c37e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:54.637596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7abef7e1523046a8970d7010ae02c37e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:54.647213 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7abef7e1523046a8970d7010ae02c37e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:54.655508 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7abef7e1523046a8970d7010ae02c37e'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:54.672774 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1424/2853] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:23:54 2022] DELETE /identity/v3/users/7abef7e1523046a8970d7010ae02c37e => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:23:54.861550 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:54.876507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa2a067-7444-461d-95e7-39956b6c2af3 service placement] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:54.933649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:54.934515 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:54.934790 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:54.935062 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] PATH_INFO: `/v3/projects/aabdb72f128d4d4da1f9390e505d704b` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:54.936288 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabdb72f128d4d4da1f9390e505d704b)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:54.938186 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:54.985376 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabdb72f128d4d4da1f9390e505d704b', 'request_id': 'req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:54.990927 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabdb72f128d4d4da1f9390e505d704b', 'request_id': 'req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:54.996973 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabdb72f128d4d4da1f9390e505d704b', 'request_id': 'req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.000475 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:23:55.014264 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] The token cache is being invalidate because project aabdb72f128d4d4da1f9390e505d704b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:23:55.014734 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.015567 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.017252 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1430/2854] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:23:54 2022] DELETE /identity/v3/projects/aabdb72f128d4d4da1f9390e505d704b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:23:55.021420 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:55.022894 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:55.036500 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd618eda-368e-4290-9a5b-f49f9b1daf05 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:55.088791 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:23:55.100449 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:55.101342 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.101654 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.101960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] PATH_INFO: `/v3/users/0c97a09c9d414424bfa71a0c78110671` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.103332 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c97a09c9d414424bfa71a0c78110671)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:55.105523 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:55.161119 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c97a09c9d414424bfa71a0c78110671'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.161798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c97a09c9d414424bfa71a0c78110671'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.162398 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c97a09c9d414424bfa71a0c78110671'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.172727 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c97a09c9d414424bfa71a0c78110671'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.180291 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-69908f73-5b6d-4887-9428-72dba2cc1708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c97a09c9d414424bfa71a0c78110671'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.200147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1425/2855] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:23:55 2022] DELETE /identity/v3/users/0c97a09c9d414424bfa71a0c78110671 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 20 09:23:55.380720 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:55.382351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:55.395825 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb60ef8-a982-40d9-a2e6-449ddc6708d5 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:55.454590 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:55.455479 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.455742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.456017 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] PATH_INFO: `/v3/projects/50db3d6e2f174362b1cf5e3d3e11eb2b` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.457241 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50db3d6e2f174362b1cf5e3d3e11eb2b)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:55.459219 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:55.507507 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50db3d6e2f174362b1cf5e3d3e11eb2b', 'request_id': 'req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.512898 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50db3d6e2f174362b1cf5e3d3e11eb2b', 'request_id': 'req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.519171 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50db3d6e2f174362b1cf5e3d3e11eb2b', 'request_id': 'req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.532814 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] The token cache is being invalidate because project 50db3d6e2f174362b1cf5e3d3e11eb2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80638) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:23:55.533100 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.533731 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-ff7feaea-972b-4d46-a6d2-6b0c9dc13578 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:23:55.535316 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1431/2856] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:23:55 2022] DELETE /identity/v3/projects/50db3d6e2f174362b1cf5e3d3e11eb2b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:23:55.540401 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd94fb-e5d3-4916-9a71-f897cb63f344 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.540918 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd94fb-e5d3-4916-9a71-f897cb63f344 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.541029 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd94fb-e5d3-4916-9a71-f897cb63f344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.559030 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-e4dd94fb-e5d3-4916-9a71-f897cb63f344 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:23:55.560253 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-e4dd94fb-e5d3-4916-9a71-f897cb63f344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:55.622612 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1426/2857] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:23:55.630675 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c9e04-dc89-4671-90c2-57e2c54339aa None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.630893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c9e04-dc89-4671-90c2-57e2c54339aa None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.631189 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08c9e04-dc89-4671-90c2-57e2c54339aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.650978 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-f08c9e04-dc89-4671-90c2-57e2c54339aa None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:23:55.652220 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f08c9e04-dc89-4671-90c2-57e2c54339aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:55.662940 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1432/2858] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:23:55.666779 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd94fb-e5d3-4916-9a71-f897cb63f344 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:55.677027 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd94fb-e5d3-4916-9a71-f897cb63f344 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:55.682757 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:55.683447 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.683738 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.684011 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] PATH_INFO: `/v3/domains` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.684747 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80637) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 20 09:23:55.685055 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:55.686672 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:55.698968 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1427/2859] 10.222.0.69 () {58 vars in 1152 bytes} [Fri May 20 09:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 20 09:23:55.703176 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c9e04-dc89-4671-90c2-57e2c54339aa None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:55.712695 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08c9e04-dc89-4671-90c2-57e2c54339aa None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:55.718492 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25320f-1fd9-45e9-978b-be808af77262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:55.719147 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25320f-1fd9-45e9-978b-be808af77262 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.719464 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25320f-1fd9-45e9-978b-be808af77262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.719738 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25320f-1fd9-45e9-978b-be808af77262 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.720428 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25320f-1fd9-45e9-978b-be808af77262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:55.722327 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25320f-1fd9-45e9-978b-be808af77262 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:55.753671 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1433/2860] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:23:55 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:23:55.758092 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:55.768979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bd1df-3535-4d07-adc3-1a4ea0551c21 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:55.827963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1b54-a8b4-44ba-88fe-723a6fde241c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:55.828639 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1b54-a8b4-44ba-88fe-723a6fde241c admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.828920 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1b54-a8b4-44ba-88fe-723a6fde241c admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.829197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820b1b54-a8b4-44ba-88fe-723a6fde241c admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.829888 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b1b54-a8b4-44ba-88fe-723a6fde241c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:55.831870 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820b1b54-a8b4-44ba-88fe-723a6fde241c admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:55.838599 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:23:55.855514 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1428/2861] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:23:55 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:23:55.859698 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25320f-1fd9-45e9-978b-be808af77262 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:55.869893 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25320f-1fd9-45e9-978b-be808af77262 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:55.874597 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0980b38-ec98-4259-8d3b-4a721feee0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:55.875275 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980b38-ec98-4259-8d3b-4a721feee0ce admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.875568 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980b38-ec98-4259-8d3b-4a721feee0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.875845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0980b38-ec98-4259-8d3b-4a721feee0ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.876581 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0980b38-ec98-4259-8d3b-4a721feee0ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:55.878207 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0980b38-ec98-4259-8d3b-4a721feee0ce admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:55.889104 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1434/2862] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:23:55.892927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1b54-a8b4-44ba-88fe-723a6fde241c admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:55.903776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820b1b54-a8b4-44ba-88fe-723a6fde241c admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:55.909528 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816ca121-1bd1-4299-b093-fa25311e4ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:55.910219 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ca121-1bd1-4299-b093-fa25311e4ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.910536 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ca121-1bd1-4299-b093-fa25311e4ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.910809 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816ca121-1bd1-4299-b093-fa25311e4ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.911726 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ca121-1bd1-4299-b093-fa25311e4ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:55.913367 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816ca121-1bd1-4299-b093-fa25311e4ae6 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:55.923752 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1429/2863] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:23:55.927910 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0980b38-ec98-4259-8d3b-4a721feee0ce admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:55.938350 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0980b38-ec98-4259-8d3b-4a721feee0ce admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:55.943509 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:55.944387 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.944669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.944949 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] PATH_INFO: `/v3/projects/0855d455651d42709d1baa57d5647015/users/d753eff4043144aa930f3b1b15058fbc/roles/580338b3f2b847c78aef559dbaaf51e4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:55.960883 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0855d455651d42709d1baa57d5647015, user_id=d753eff4043144aa930f3b1b15058fbc, role_id=580338b3f2b847c78aef559dbaaf51e4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:55.963632 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:55.982795 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1435/2864] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:23:55 2022] PUT /identity/v3/projects/0855d455651d42709d1baa57d5647015/users/d753eff4043144aa930f3b1b15058fbc/roles/580338b3f2b847c78aef559dbaaf51e4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:23:55.987954 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61222585-9505-4551-bad6-a67e7e891b01 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:55.988271 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61222585-9505-4551-bad6-a67e7e891b01 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:55.988543 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61222585-9505-4551-bad6-a67e7e891b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:56.006927 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-61222585-9505-4551-bad6-a67e7e891b01 None None] MFA Rules not processed for user `ce1d4da72fcf4acabf4a7749465fe917`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:23:56.008161 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-61222585-9505-4551-bad6-a67e7e891b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:56.068438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1430/2865] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:23:56.081465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:56.084532 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:56.111604 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:23:56.149369 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893823b6-b49b-465d-8141-2918e8211d8f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:56.153263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ea07e-7125-46f8-ace4-bb47f78be3e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:56.153909 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea07e-7125-46f8-ace4-bb47f78be3e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:56.154192 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea07e-7125-46f8-ace4-bb47f78be3e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:56.154467 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ea07e-7125-46f8-ace4-bb47f78be3e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:56.163939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ea07e-7125-46f8-ace4-bb47f78be3e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:56.165373 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301ea07e-7125-46f8-ace4-bb47f78be3e3 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:56.174919 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1436/2866] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:23:57.979104 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61222585-9505-4551-bad6-a67e7e891b01 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:57.989572 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-61222585-9505-4551-bad6-a67e7e891b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:58.091172 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61222585-9505-4551-bad6-a67e7e891b01 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:58.097478 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0986df1a-39d6-4d88-9f4b-779ef696b482 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:58.098149 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986df1a-39d6-4d88-9f4b-779ef696b482 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:58.098443 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986df1a-39d6-4d88-9f4b-779ef696b482 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:58.099242 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0986df1a-39d6-4d88-9f4b-779ef696b482 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:58.100716 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-0986df1a-39d6-4d88-9f4b-779ef696b482 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:58.118090 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986df1a-39d6-4d88-9f4b-779ef696b482 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:58.120298 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0986df1a-39d6-4d88-9f4b-779ef696b482 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:58.134518 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1431/2867] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:23:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:23:59.195112 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ea07e-7125-46f8-ace4-bb47f78be3e3 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:23:59.233831 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ea07e-7125-46f8-ace4-bb47f78be3e3 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:23:59.252593 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:23:59.253277 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:23:59.253563 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:23:59.253843 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:23:59.261827 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:23:59.298961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:23:59.300527 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:23:59.313767 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1437/2868] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:23:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:24:00.861343 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42c0a-cb22-40c7-8ec2-c54d50a629c4 None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:00.861835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42c0a-cb22-40c7-8ec2-c54d50a629c4 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:00.861996 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42c0a-cb22-40c7-8ec2-c54d50a629c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:00.893178 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-f9d42c0a-cb22-40c7-8ec2-c54d50a629c4 None None] MFA Rules not processed for user `d753eff4043144aa930f3b1b15058fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:24:00.894421 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f9d42c0a-cb22-40c7-8ec2-c54d50a629c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:00.979252 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:00.991947 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1432/2869] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:24:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:24:00.996232 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:01.007514 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e6962-b4cc-45b3-8f7c-0612d7d66f18 service neutron] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:01.013544 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65371a9e-d941-4997-8173-070f12faccf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:01.014271 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65371a9e-d941-4997-8173-070f12faccf4 admin admin] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:01.014551 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65371a9e-d941-4997-8173-070f12faccf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:01.015087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65371a9e-d941-4997-8173-070f12faccf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:01.015841 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65371a9e-d941-4997-8173-070f12faccf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:01.017958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65371a9e-d941-4997-8173-070f12faccf4 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:01.046140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1438/2870] 10.222.0.69 () {60 vars in 1152 bytes} [Fri May 20 09:24:00 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 20 09:24:01.050174 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d42c0a-cb22-40c7-8ec2-c54d50a629c4 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:01.061022 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d42c0a-cb22-40c7-8ec2-c54d50a629c4 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:01.117280 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a858d26-fd38-481b-9db9-3392dfe9438f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:01.117961 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858d26-fd38-481b-9db9-3392dfe9438f admin admin] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:01.118247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858d26-fd38-481b-9db9-3392dfe9438f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:01.118525 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a858d26-fd38-481b-9db9-3392dfe9438f admin admin] PATH_INFO: `/v3/users` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:01.119229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a858d26-fd38-481b-9db9-3392dfe9438f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:01.121207 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a858d26-fd38-481b-9db9-3392dfe9438f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:01.143462 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1433/2871] 10.222.0.69 () {60 vars in 1143 bytes} [Fri May 20 09:24:01 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 20 09:24:01.147890 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65371a9e-d941-4997-8173-070f12faccf4 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:01.158569 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65371a9e-d941-4997-8173-070f12faccf4 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:01.163802 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dee2c3-f7b3-431a-abd6-4054c8aa32bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:01.165417 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dee2c3-f7b3-431a-abd6-4054c8aa32bb admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:01.166506 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dee2c3-f7b3-431a-abd6-4054c8aa32bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:01.166963 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dee2c3-f7b3-431a-abd6-4054c8aa32bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:01.167939 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dee2c3-f7b3-431a-abd6-4054c8aa32bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:01.169872 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dee2c3-f7b3-431a-abd6-4054c8aa32bb admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:01.183997 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1439/2872] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:24:01.188001 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a858d26-fd38-481b-9db9-3392dfe9438f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:01.199472 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a858d26-fd38-481b-9db9-3392dfe9438f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:01.205573 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c8b38-5053-4528-84f5-76a4d8d0bdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:01.206481 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c8b38-5053-4528-84f5-76a4d8d0bdaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:01.206764 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c8b38-5053-4528-84f5-76a4d8d0bdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:01.207039 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c8b38-5053-4528-84f5-76a4d8d0bdaa admin admin] PATH_INFO: `/v3/projects/d4a5d6e8c7c54171955700b2c1fadafc/users/f6ab63f377fc48dc982bf51340ead7eb/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:01.223810 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c8b38-5053-4528-84f5-76a4d8d0bdaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a5d6e8c7c54171955700b2c1fadafc, user_id=f6ab63f377fc48dc982bf51340ead7eb, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:01.226527 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c8b38-5053-4528-84f5-76a4d8d0bdaa admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:01.241146 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1434/2873] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:24:01 2022] PUT /identity/v3/projects/d4a5d6e8c7c54171955700b2c1fadafc/users/f6ab63f377fc48dc982bf51340ead7eb/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:24:01.245180 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dee2c3-f7b3-431a-abd6-4054c8aa32bb admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:01.255838 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dee2c3-f7b3-431a-abd6-4054c8aa32bb admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:01.317513 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f72f4b-6c8b-45cb-ae04-d964f1759f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:01.318185 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72f4b-6c8b-45cb-ae04-d964f1759f22 admin admin] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:01.318516 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72f4b-6c8b-45cb-ae04-d964f1759f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:01.318812 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f72f4b-6c8b-45cb-ae04-d964f1759f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:01.319576 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f72f4b-6c8b-45cb-ae04-d964f1759f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:01.321172 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f72f4b-6c8b-45cb-ae04-d964f1759f22 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:01.331047 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1440/2874] 10.222.0.69 () {58 vars in 1121 bytes} [Fri May 20 09:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 20 09:24:01.335937 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c8b38-5053-4528-84f5-76a4d8d0bdaa admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:01.346957 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c8b38-5053-4528-84f5-76a4d8d0bdaa admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:01.352438 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:01.353623 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:01.353776 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:01.354150 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] PATH_INFO: `/v3/projects/d4a5d6e8c7c54171955700b2c1fadafc/users/f6ab63f377fc48dc982bf51340ead7eb/roles/6dca675ade814980a252f800fb4fa3ad` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:01.356491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4a5d6e8c7c54171955700b2c1fadafc, user_id=f6ab63f377fc48dc982bf51340ead7eb, role_id=6dca675ade814980a252f800fb4fa3ad)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:01.359608 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:01.373697 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1435/2875] 10.222.0.69 () {60 vars in 1482 bytes} [Fri May 20 09:24:01 2022] PUT /identity/v3/projects/d4a5d6e8c7c54171955700b2c1fadafc/users/f6ab63f377fc48dc982bf51340ead7eb/roles/6dca675ade814980a252f800fb4fa3ad => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:24:05.993409 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651cbb-818d-45fd-836e-7c22e2844638 None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:05.994873 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651cbb-818d-45fd-836e-7c22e2844638 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:05.995249 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11651cbb-818d-45fd-836e-7c22e2844638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:06.058263 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-11651cbb-818d-45fd-836e-7c22e2844638 None None] MFA Rules not processed for user `f6ab63f377fc48dc982bf51340ead7eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:24:06.063025 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-11651cbb-818d-45fd-836e-7c22e2844638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:06.202833 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:06.209601 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1441/2876] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:24:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:24:06.222997 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:06.226537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:06.368677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa62dc-8eb7-4022-b136-f553ade08be7 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:06.375111 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74423420-5086-4c49-b7d7-92425a37f7f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:06.375874 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74423420-5086-4c49-b7d7-92425a37f7f0 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:06.376209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74423420-5086-4c49-b7d7-92425a37f7f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:06.376491 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74423420-5086-4c49-b7d7-92425a37f7f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:06.432794 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:06.453299 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74423420-5086-4c49-b7d7-92425a37f7f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:06.457690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74423420-5086-4c49-b7d7-92425a37f7f0 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:06.469205 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1436/2877] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:24:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:24:07.938140 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6ad90-40a7-4926-ac4c-7e30f5a1163e None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:07.938649 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6ad90-40a7-4926-ac4c-7e30f5a1163e None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:07.938747 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6ad90-40a7-4926-ac4c-7e30f5a1163e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:07.956938 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-2ab6ad90-40a7-4926-ac4c-7e30f5a1163e None None] MFA Rules not processed for user `d753eff4043144aa930f3b1b15058fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:24:07.958087 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2ab6ad90-40a7-4926-ac4c-7e30f5a1163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:07.967848 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1442/2878] 10.222.0.69 () {58 vars in 957 bytes} [Fri May 20 09:24:07 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:24:11.750043 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74423420-5086-4c49-b7d7-92425a37f7f0 service nova] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:11.810437 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74423420-5086-4c49-b7d7-92425a37f7f0 service nova] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:11.813960 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:11.814616 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:11.814898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:11.815354 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:11.817139 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:11.887798 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:11.889660 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:11.902758 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1437/2879] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:24:17.951571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=3 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:18.214732 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6ad90-40a7-4926-ac4c-7e30f5a1163e None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:18.216794 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2ab6ad90-40a7-4926-ac4c-7e30f5a1163e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:18.236801 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6ad90-40a7-4926-ac4c-7e30f5a1163e None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:18.241929 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:18.242787 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:18.243073 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:18.243410 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] PATH_INFO: `/v3/users/bc0307909d614a1f97f006dc6d091cd4` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:18.244677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc0307909d614a1f97f006dc6d091cd4)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:18.246627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:18.300046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc0307909d614a1f97f006dc6d091cd4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.307739 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc0307909d614a1f97f006dc6d091cd4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.308571 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc0307909d614a1f97f006dc6d091cd4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.322904 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc0307909d614a1f97f006dc6d091cd4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.323545 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc0307909d614a1f97f006dc6d091cd4'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.341960 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1443/2880] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:24:18 2022] DELETE /identity/v3/users/bc0307909d614a1f97f006dc6d091cd4 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:24:18.518005 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:18.529538 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d9da8-0845-44fc-bd73-2c90cc22a598 service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:18.588109 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:18.588936 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:18.589199 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:18.589471 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] PATH_INFO: `/v3/projects/f26ac8782fe0407cb6f146ba70e74246` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:18.590710 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f26ac8782fe0407cb6f146ba70e74246)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:18.592661 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:18.632932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:18.645690 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f26ac8782fe0407cb6f146ba70e74246', 'request_id': 'req-05520318-62c9-435f-9430-ba8e3fa14114'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.657377 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f26ac8782fe0407cb6f146ba70e74246', 'request_id': 'req-05520318-62c9-435f-9430-ba8e3fa14114'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.663898 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f26ac8782fe0407cb6f146ba70e74246', 'request_id': 'req-05520318-62c9-435f-9430-ba8e3fa14114'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.674818 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] The token cache is being invalidate because project f26ac8782fe0407cb6f146ba70e74246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:24:18.675107 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.675788 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.677348 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1438/2881] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:24:18 2022] DELETE /identity/v3/projects/f26ac8782fe0407cb6f146ba70e74246 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:24:18.683918 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:18.686697 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:18.701709 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe45161-1665-4319-8cf0-e591ed7b2ecd admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:18.764351 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:18.765224 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:18.765494 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:18.765764 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] PATH_INFO: `/v3/users/fddb4b4558dd47b9925995a565bf9fd3` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:18.766942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fddb4b4558dd47b9925995a565bf9fd3)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:18.768917 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:18.822364 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fddb4b4558dd47b9925995a565bf9fd3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.829887 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fddb4b4558dd47b9925995a565bf9fd3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.830669 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fddb4b4558dd47b9925995a565bf9fd3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.841462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fddb4b4558dd47b9925995a565bf9fd3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.842169 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fddb4b4558dd47b9925995a565bf9fd3'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:18.850132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:18.861777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1444/2882] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:24:18 2022] DELETE /identity/v3/users/fddb4b4558dd47b9925995a565bf9fd3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:24:19.043033 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:19.044677 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:19.058924 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05520318-62c9-435f-9430-ba8e3fa14114 admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:19.116448 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:19.117325 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:19.117655 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:19.117932 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] PATH_INFO: `/v3/projects/d05f58af207e431eac6b187af266d58d` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:19.119243 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d05f58af207e431eac6b187af266d58d)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:19.121197 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:19.170246 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd05f58af207e431eac6b187af266d58d', 'request_id': 'req-4430a5e7-0dad-4c53-b045-22464874377f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:19.179048 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd05f58af207e431eac6b187af266d58d', 'request_id': 'req-4430a5e7-0dad-4c53-b045-22464874377f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:19.185021 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd05f58af207e431eac6b187af266d58d', 'request_id': 'req-4430a5e7-0dad-4c53-b045-22464874377f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:19.196619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] The token cache is being invalidate because project d05f58af207e431eac6b187af266d58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:24:19.196619 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:19.197125 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:24:19.198928 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1439/2883] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:24:19 2022] DELETE /identity/v3/projects/d05f58af207e431eac6b187af266d58d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:24:23.009046 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:23.010687 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:23.083673 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2380e81-5f1d-45cb-b7b3-c918a438794f admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:23.143583 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:23.144285 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:23.144566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:23.144840 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:23.147029 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:23.158905 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:23.160566 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:23.170707 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1445/2884] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:24:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:24:29.193383 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:DELETE=12,keystone:SELECT=62,keystone:INSERT=2 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:33.166543 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=61,keystone:DELETE=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:36.987684 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:36.989366 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:37.065244 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a5e7-0dad-4c53-b045-22464874377f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:37.070903 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '04cd27bb637a404d827d384196eabfb1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:37.071593 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:37.071885 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:37.072160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:37.074247 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:37.086610 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:37.088239 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:37.098778 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1440/2885] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:24:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:24:38.897696 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:24:38.899221 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:38.955760 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec58a5-0014-467b-828c-e142bf39f135 service placement] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:24:38.961488 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f5d11-493c-4d2a-b2d8-cb065ff30231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:24:38.962462 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f5d11-493c-4d2a-b2d8-cb065ff30231 service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:24:38.962875 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f5d11-493c-4d2a-b2d8-cb065ff30231 service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:24:38.963295 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f5d11-493c-4d2a-b2d8-cb065ff30231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:24:38.966530 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-d11f5d11-493c-4d2a-b2d8-cb065ff30231 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:24:39.037390 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f5d11-493c-4d2a-b2d8-cb065ff30231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:24:39.038845 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f5d11-493c-4d2a-b2d8-cb065ff30231 service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:24:39.048866 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1446/2886] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:24:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:24:47.094495 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=34 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:24:49.044676 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=50 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:26:49.243214 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:26:49.244833 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:26:49.311312 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5e8d28-1081-4a85-ad61-fea54bca4604 service nova] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:26:49.315502 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a555d59-9936-4318-9a6d-c470eb4dbe20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:26:49.316160 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a555d59-9936-4318-9a6d-c470eb4dbe20 service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:26:49.316441 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a555d59-9936-4318-9a6d-c470eb4dbe20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:26:49.316714 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a555d59-9936-4318-9a6d-c470eb4dbe20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:26:49.318695 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-8a555d59-9936-4318-9a6d-c470eb4dbe20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:26:49.328539 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a555d59-9936-4318-9a6d-c470eb4dbe20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:26:49.329989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a555d59-9936-4318-9a6d-c470eb4dbe20 service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:26:49.338218 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1441/2887] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:26:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:26:49.451613 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c6038-1284-4336-b1c0-9bd4a5f4bb60 None None] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:26:49.452211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c6038-1284-4336-b1c0-9bd4a5f4bb60 None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:26:49.452211 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c6038-1284-4336-b1c0-9bd4a5f4bb60 None None] PATH_INFO: `/` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:26:49.453627 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1447/2888] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:26:49.459664 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d845c7-1486-4cd9-ae40-15caff3829bf None None] REQUEST_METHOD: `POST` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:26:49.459926 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d845c7-1486-4cd9-ae40-15caff3829bf None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:26:49.460209 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d845c7-1486-4cd9-ae40-15caff3829bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:26:49.522076 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.auth.core [None req-72d845c7-1486-4cd9-ae40-15caff3829bf None None] MFA Rules not processed for user `349e99511c7c4307b27dc90b01f2dcd3`. Rule list: `[]` (Enabled: `True`). {{(pid=80637) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:26:49.523295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-72d845c7-1486-4cd9-ae40-15caff3829bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:26:49.530288 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1442/2889] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:26:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:26:49.542741 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c6038-1284-4336-b1c0-9bd4a5f4bb60 None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:26:49.553816 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c6038-1284-4336-b1c0-9bd4a5f4bb60 None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:26:49.557459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b218f3-ffed-44b9-bf88-6e8f096aecee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:26:49.558106 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b218f3-ffed-44b9-bf88-6e8f096aecee service neutron] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:26:49.558389 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b218f3-ffed-44b9-bf88-6e8f096aecee service neutron] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:26:49.558666 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b218f3-ffed-44b9-bf88-6e8f096aecee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:26:49.566677 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b218f3-ffed-44b9-bf88-6e8f096aecee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:26:49.568132 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b218f3-ffed-44b9-bf88-6e8f096aecee service neutron] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:26:49.575958 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1448/2890] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:26:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:26:49.659416 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefb425-2147-4d15-8566-3af2e42d3d38 None None] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:26:49.659706 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefb425-2147-4d15-8566-3af2e42d3d38 None None] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:26:49.660007 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aefb425-2147-4d15-8566-3af2e42d3d38 None None] PATH_INFO: `/` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:26:49.661382 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1443/2891] 10.222.0.69 () {54 vars in 921 bytes} [Fri May 20 09:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 20 09:26:49.668375 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711402fa-50f7-401b-8556-da8981d9001d None None] REQUEST_METHOD: `POST` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:26:49.668742 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711402fa-50f7-401b-8556-da8981d9001d None None] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:26:49.669101 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711402fa-50f7-401b-8556-da8981d9001d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:26:49.725060 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.auth.core [None req-711402fa-50f7-401b-8556-da8981d9001d None None] MFA Rules not processed for user `04cd27bb637a404d827d384196eabfb1`. Rule list: `[]` (Enabled: `True`). {{(pid=80638) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 20 09:26:49.726266 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-711402fa-50f7-401b-8556-da8981d9001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:26:49.776184 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1449/2892] 10.222.0.69 () {58 vars in 1018 bytes} [Fri May 20 09:26:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 20 09:26:49.789079 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefb425-2147-4d15-8566-3af2e42d3d38 None None] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:26:49.800596 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aefb425-2147-4d15-8566-3af2e42d3d38 None None] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:26:49.804963 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb5f87-348a-41f1-bcfd-11f73eacb0af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '349e99511c7c4307b27dc90b01f2dcd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:26:49.805627 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb5f87-348a-41f1-bcfd-11f73eacb0af service neutron] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:26:49.805905 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb5f87-348a-41f1-bcfd-11f73eacb0af service neutron] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:26:49.806181 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceb5f87-348a-41f1-bcfd-11f73eacb0af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:26:49.817102 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb5f87-348a-41f1-bcfd-11f73eacb0af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:26:49.818566 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ceb5f87-348a-41f1-bcfd-11f73eacb0af service neutron] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:26:49.829980 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1444/2893] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:26:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:26:59.772643 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=36 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:26:59.825141 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=51 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:27:32.306234 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711402fa-50f7-401b-8556-da8981d9001d None None] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:27:32.307989 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-711402fa-50f7-401b-8556-da8981d9001d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:27:32.346306 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711402fa-50f7-401b-8556-da8981d9001d None None] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:27:32.352102 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:27:32.352772 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:27:32.353050 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:27:32.353320 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:27:32.355317 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:27:32.366717 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:27:32.368356 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:27:32.378334 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1450/2894] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:27:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:27:42.374330 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=15 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:28:17.524753 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb5f87-348a-41f1-bcfd-11f73eacb0af service neutron] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:28:17.538948 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceb5f87-348a-41f1-bcfd-11f73eacb0af service neutron] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:28:17.545621 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:28:17.546282 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] REQUEST_METHOD: `GET` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:28:17.546614 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:28:17.546835 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:28:17.548766 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:28:17.563085 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:28:17.564768 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:28:17.583979 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1445/2895] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:28:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:28:20.504239 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:28:20.505799 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:28:20.608034 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8b2dc-ab81-4754-aa9f-fa2cff8d31ae service placement] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:28:20.612562 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:28:20.613480 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:28:20.613758 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:28:20.614031 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] PATH_INFO: `/v3/users/d753eff4043144aa930f3b1b15058fbc` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:28:20.615199 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d753eff4043144aa930f3b1b15058fbc)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:28:20.617166 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:28:20.668125 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd753eff4043144aa930f3b1b15058fbc'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:20.674913 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd753eff4043144aa930f3b1b15058fbc'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:20.675647 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd753eff4043144aa930f3b1b15058fbc'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:20.686312 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd753eff4043144aa930f3b1b15058fbc'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:20.686942 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd753eff4043144aa930f3b1b15058fbc'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:20.703630 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1451/2896] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:28:20 2022] DELETE /identity/v3/users/d753eff4043144aa930f3b1b15058fbc => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:28:20.894711 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:28:20.906995 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a17f251-fdfa-416c-a66b-358941babb8c service placement] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:28:20.968388 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:28:20.969259 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:28:20.969537 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:28:20.969804 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] PATH_INFO: `/v3/projects/0855d455651d42709d1baa57d5647015` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:28:20.970989 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0855d455651d42709d1baa57d5647015)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:28:20.972969 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:28:21.020563 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0855d455651d42709d1baa57d5647015', 'request_id': 'req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.031772 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0855d455651d42709d1baa57d5647015', 'request_id': 'req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.037908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0855d455651d42709d1baa57d5647015', 'request_id': 'req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.049855 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] The token cache is being invalidate because project 0855d455651d42709d1baa57d5647015 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:28:21.050147 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.050871 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.052632 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1446/2897] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:28:20 2022] DELETE /identity/v3/projects/0855d455651d42709d1baa57d5647015 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:28:21.057036 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:28:21.058459 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:28:21.071922 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29782e-f1a4-43b2-b67f-4298b8495116 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:28:21.120961 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:28:21.132426 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:28:21.133465 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:28:21.133777 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:28:21.134065 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] PATH_INFO: `/v3/users/f6ab63f377fc48dc982bf51340ead7eb` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:28:21.135784 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6ab63f377fc48dc982bf51340ead7eb)` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:28:21.137959 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:28:21.187246 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6ab63f377fc48dc982bf51340ead7eb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.193962 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6ab63f377fc48dc982bf51340ead7eb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.194605 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6ab63f377fc48dc982bf51340ead7eb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.204128 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6ab63f377fc48dc982bf51340ead7eb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.204892 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.notifications [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6ab63f377fc48dc982bf51340ead7eb'} {{(pid=80638) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.222214 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1452/2898] 10.222.0.69 () {58 vars in 1223 bytes} [Fri May 20 09:28:21 2022] DELETE /identity/v3/users/f6ab63f377fc48dc982bf51340ead7eb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:28:21.402908 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] Authenticating user token {{(pid=80637) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:28:21.405078 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.fernet_utils [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80637) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:28:21.422302 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847e5a7-a67a-4a28-9233-6ad5ff8d865f admin admin] Validating token access rules against request {{(pid=80637) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:28:21.482327 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1d4da72fcf4acabf4a7749465fe917', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'aac440dc4ce14fa1a6d365f33d6d214c', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80637) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:28:21.483852 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:28:21.484131 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:28:21.484399 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] PATH_INFO: `/v3/projects/d4a5d6e8c7c54171955700b2c1fadafc` {{(pid=80637) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:28:21.485615 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4a5d6e8c7c54171955700b2c1fadafc)` {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:28:21.487587 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] RBAC: Authorization granted {{(pid=80637) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:28:21.508153 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:28:21.535235 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4a5d6e8c7c54171955700b2c1fadafc', 'request_id': 'req-94be8139-1e2c-40f1-8de1-333705b5a819'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.546329 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4a5d6e8c7c54171955700b2c1fadafc', 'request_id': 'req-94be8139-1e2c-40f1-8de1-333705b5a819'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.552023 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4a5d6e8c7c54171955700b2c1fadafc', 'request_id': 'req-94be8139-1e2c-40f1-8de1-333705b5a819'} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.563229 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] The token cache is being invalidate because project d4a5d6e8c7c54171955700b2c1fadafc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80637) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 20 09:28:21.563556 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.564295 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG keystone.notifications [None req-94be8139-1e2c-40f1-8de1-333705b5a819 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80637) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 20 09:28:21.565965 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: [pid: 80637|app: 0|req: 1447/2899] 10.222.0.69 () {58 vars in 1232 bytes} [Fri May 20 09:28:21 2022] DELETE /identity/v3/projects/d4a5d6e8c7c54171955700b2c1fadafc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 20 09:28:31.219236 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=30,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:28:31.560185 dv-cinder-836996-23-iscsi devstack@keystone.service[80637]: DEBUG dbcounter [-] [80637] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80637) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 20 09:32:35.595481 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] Authenticating user token {{(pid=80638) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 20 09:32:35.597085 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:32:35.670075 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246631a-20e6-4f34-ad68-6996cb777a72 admin admin] Validating token access rules against request {{(pid=80638) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 20 09:32:35.733540 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134b6b6-f978-418b-baa4-aba2ee5bc124 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6def7fd117dc4b7598a1864e63f09427', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '569782752acb4085ad742330af999bc2', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80638) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 20 09:32:35.734198 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b6b6-f978-418b-baa4-aba2ee5bc124 service placement] REQUEST_METHOD: `GET` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 20 09:32:35.734489 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b6b6-f978-418b-baa4-aba2ee5bc124 service placement] SCRIPT_NAME: `/identity` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 20 09:32:35.734765 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134b6b6-f978-418b-baa4-aba2ee5bc124 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80638) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 20 09:32:35.736635 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.fernet_utils [None req-2134b6b6-f978-418b-baa4-aba2ee5bc124 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80638) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 20 09:32:35.748121 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134b6b6-f978-418b-baa4-aba2ee5bc124 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 20 09:32:35.749645 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134b6b6-f978-418b-baa4-aba2ee5bc124 service placement] RBAC: Authorization granted {{(pid=80638) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 20 09:32:35.760008 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: [pid: 80638|app: 0|req: 1453/2900] 10.222.0.69 () {60 vars in 1365 bytes} [Fri May 20 09:32:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 20 09:32:45.756071 dv-cinder-836996-23-iscsi devstack@keystone.service[80638]: DEBUG dbcounter [-] [80638] Writing DB stats keystone:SELECT=58 {{(pid=80638) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}